Credit Card Fraud Detection
PROJECT TOPIC - CREDIT CARD FRAUD DETECTION
#1. Data Description: Time- Number of seconds elapsed between this transaction and the first transaction in the dataset V1-V28 - Result of a PCA Dimensionality reduction to protect user identities and sensitive features(v1-v28) Amount - Transaction amount Class - 1 for fraudulent transactions, 0 otherwise
#Goal
To identify fraudulent transactions which can result in significant financial losses for individuals and businesses. The goal is to develop a machine learning model that can accurately classify transactions as either fraudulent or legitimate, based on various features such as transaction amount, location, and time.
#Dataset Reference:
The Credit Card Fraud Detection dataset on Kaggle at the following link https://www.kaggle.com/mlg-ulb/creditcardfraud Links to an external site. The dataset has 31 variables, with the target variable indicating whether a transaction is fraudulent or not.
#Model:
I will be using a classification model to solve the problem of credit card fraud detection. The outcome variable will be a binary variable indicating whether a credit card transaction is fraudulent or not.
library(tidyverse)
library(caret)
library(ggplot2)
library(lattice)
library(gridExtra)
library(corrplot)
library(ROCR)#1 Loading the dataset:
credit_card <- read.csv("/Users/vinoth/Downloads/creditcard.csv")
credit_card$Class = factor(credit_card$Class,levels = c(0,1))Getting the summary of structure of dataset
head(credit_card)dim(credit_card)[1] 284807 31str(credit_card)'data.frame': 284807 obs. of 31 variables:
$ Time : num 0 0 1 1 2 2 4 7 7 9 ...
$ V1 : num -1.36 1.192 -1.358 -0.966 -1.158 ...
$ V2 : num -0.0728 0.2662 -1.3402 -0.1852 0.8777 ...
$ V3 : num 2.536 0.166 1.773 1.793 1.549 ...
$ V4 : num 1.378 0.448 0.38 -0.863 0.403 ...
$ V5 : num -0.3383 0.06 -0.5032 -0.0103 -0.4072 ...
$ V6 : num 0.4624 -0.0824 1.8005 1.2472 0.0959 ...
$ V7 : num 0.2396 -0.0788 0.7915 0.2376 0.5929 ...
$ V8 : num 0.0987 0.0851 0.2477 0.3774 -0.2705 ...
$ V9 : num 0.364 -0.255 -1.515 -1.387 0.818 ...
$ V10 : num 0.0908 -0.167 0.2076 -0.055 0.7531 ...
$ V11 : num -0.552 1.613 0.625 -0.226 -0.823 ...
$ V12 : num -0.6178 1.0652 0.0661 0.1782 0.5382 ...
$ V13 : num -0.991 0.489 0.717 0.508 1.346 ...
$ V14 : num -0.311 -0.144 -0.166 -0.288 -1.12 ...
$ V15 : num 1.468 0.636 2.346 -0.631 0.175 ...
$ V16 : num -0.47 0.464 -2.89 -1.06 -0.451 ...
$ V17 : num 0.208 -0.115 1.11 -0.684 -0.237 ...
$ V18 : num 0.0258 -0.1834 -0.1214 1.9658 -0.0382 ...
$ V19 : num 0.404 -0.146 -2.262 -1.233 0.803 ...
$ V20 : num 0.2514 -0.0691 0.525 -0.208 0.4085 ...
$ V21 : num -0.01831 -0.22578 0.248 -0.1083 -0.00943 ...
$ V22 : num 0.27784 -0.63867 0.77168 0.00527 0.79828 ...
$ V23 : num -0.11 0.101 0.909 -0.19 -0.137 ...
$ V24 : num 0.0669 -0.3398 -0.6893 -1.1756 0.1413 ...
$ V25 : num 0.129 0.167 -0.328 0.647 -0.206 ...
$ V26 : num -0.189 0.126 -0.139 -0.222 0.502 ...
$ V27 : num 0.13356 -0.00898 -0.05535 0.06272 0.21942 ...
$ V28 : num -0.0211 0.0147 -0.0598 0.0615 0.2152 ...
$ Amount: num 149.62 2.69 378.66 123.5 69.99 ...
$ Class : Factor w/ 2 levels "0","1": 1 1 1 1 1 1 1 1 1 1 ...summary(credit_card) Time V1 V2 V3 V4 V5
Min. : 0 Min. :-56.40751 Min. :-72.71573 Min. :-48.3256 Min. :-5.68317 Min. :-113.74331
1st Qu.: 54202 1st Qu.: -0.92037 1st Qu.: -0.59855 1st Qu.: -0.8904 1st Qu.:-0.84864 1st Qu.: -0.69160
Median : 84692 Median : 0.01811 Median : 0.06549 Median : 0.1799 Median :-0.01985 Median : -0.05434
Mean : 94814 Mean : 0.00000 Mean : 0.00000 Mean : 0.0000 Mean : 0.00000 Mean : 0.00000
3rd Qu.:139320 3rd Qu.: 1.31564 3rd Qu.: 0.80372 3rd Qu.: 1.0272 3rd Qu.: 0.74334 3rd Qu.: 0.61193
Max. :172792 Max. : 2.45493 Max. : 22.05773 Max. : 9.3826 Max. :16.87534 Max. : 34.80167
V6 V7 V8 V9 V10 V11
Min. :-26.1605 Min. :-43.5572 Min. :-73.21672 Min. :-13.43407 Min. :-24.58826 Min. :-4.79747
1st Qu.: -0.7683 1st Qu.: -0.5541 1st Qu.: -0.20863 1st Qu.: -0.64310 1st Qu.: -0.53543 1st Qu.:-0.76249
Median : -0.2742 Median : 0.0401 Median : 0.02236 Median : -0.05143 Median : -0.09292 Median :-0.03276
Mean : 0.0000 Mean : 0.0000 Mean : 0.00000 Mean : 0.00000 Mean : 0.00000 Mean : 0.00000
3rd Qu.: 0.3986 3rd Qu.: 0.5704 3rd Qu.: 0.32735 3rd Qu.: 0.59714 3rd Qu.: 0.45392 3rd Qu.: 0.73959
Max. : 73.3016 Max. :120.5895 Max. : 20.00721 Max. : 15.59500 Max. : 23.74514 Max. :12.01891
V12 V13 V14 V15 V16 V17
Min. :-18.6837 Min. :-5.79188 Min. :-19.2143 Min. :-4.49894 Min. :-14.12985 Min. :-25.16280
1st Qu.: -0.4056 1st Qu.:-0.64854 1st Qu.: -0.4256 1st Qu.:-0.58288 1st Qu.: -0.46804 1st Qu.: -0.48375
Median : 0.1400 Median :-0.01357 Median : 0.0506 Median : 0.04807 Median : 0.06641 Median : -0.06568
Mean : 0.0000 Mean : 0.00000 Mean : 0.0000 Mean : 0.00000 Mean : 0.00000 Mean : 0.00000
3rd Qu.: 0.6182 3rd Qu.: 0.66251 3rd Qu.: 0.4931 3rd Qu.: 0.64882 3rd Qu.: 0.52330 3rd Qu.: 0.39968
Max. : 7.8484 Max. : 7.12688 Max. : 10.5268 Max. : 8.87774 Max. : 17.31511 Max. : 9.25353
V18 V19 V20 V21 V22
Min. :-9.498746 Min. :-7.213527 Min. :-54.49772 Min. :-34.83038 Min. :-10.933144
1st Qu.:-0.498850 1st Qu.:-0.456299 1st Qu.: -0.21172 1st Qu.: -0.22839 1st Qu.: -0.542350
Median :-0.003636 Median : 0.003735 Median : -0.06248 Median : -0.02945 Median : 0.006782
Mean : 0.000000 Mean : 0.000000 Mean : 0.00000 Mean : 0.00000 Mean : 0.000000
3rd Qu.: 0.500807 3rd Qu.: 0.458949 3rd Qu.: 0.13304 3rd Qu.: 0.18638 3rd Qu.: 0.528554
Max. : 5.041069 Max. : 5.591971 Max. : 39.42090 Max. : 27.20284 Max. : 10.503090
V23 V24 V25 V26 V27
Min. :-44.80774 Min. :-2.83663 Min. :-10.29540 Min. :-2.60455 Min. :-22.565679
1st Qu.: -0.16185 1st Qu.:-0.35459 1st Qu.: -0.31715 1st Qu.:-0.32698 1st Qu.: -0.070840
Median : -0.01119 Median : 0.04098 Median : 0.01659 Median :-0.05214 Median : 0.001342
Mean : 0.00000 Mean : 0.00000 Mean : 0.00000 Mean : 0.00000 Mean : 0.000000
3rd Qu.: 0.14764 3rd Qu.: 0.43953 3rd Qu.: 0.35072 3rd Qu.: 0.24095 3rd Qu.: 0.091045
Max. : 22.52841 Max. : 4.58455 Max. : 7.51959 Max. : 3.51735 Max. : 31.612198
V28 Amount Class
Min. :-15.43008 Min. : 0.00 0:284315
1st Qu.: -0.05296 1st Qu.: 5.60 1: 492
Median : 0.01124 Median : 22.00
Mean : 0.00000 Mean : 88.35
3rd Qu.: 0.07828 3rd Qu.: 77.17
Max. : 33.84781 Max. :25691.16 Using duplicated function to duplicated elements.
duplicated(credit_card) [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[19] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE
[37] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[55] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[73] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[91] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[109] FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[127] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[145] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[163] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[181] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[199] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[217] FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[235] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[253] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[271] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[289] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[307] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[325] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[343] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[361] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[379] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[397] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[415] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[433] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[451] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[469] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[487] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[505] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[523] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[541] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[559] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[577] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[595] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[613] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[631] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[649] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[667] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[685] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[703] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[721] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[739] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[757] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[775] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[793] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[811] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[829] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[847] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[865] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[883] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[901] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[919] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[937] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[955] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[973] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[991] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[ reached getOption("max.print") -- omitted 283807 entries ]#Number of missing values in realtor dataset.
sum(is.na(credit_card))[1] 0There are no missing values in the dataset.
#Class Distribution:
table(credit_card$Class)
0 1
284315 492 barplot(table(credit_card$Class))
#Time and Amount Analysis:
hist(credit_card$Time)
hist(credit_card$Amount)
boxplot(Amount ~ Class, data = credit_card, xlab = "Class", ylab = "Amount")
plot(credit_card$V1, credit_card$V2, col = ifelse(credit_card$Class == 1, "red", "blue"),xlab = "V1", ylab = "V2")
#Finding the Correlation of every feature vs Class Feature.
correlation <- cor(credit_card[,-31], as.numeric(credit_card[,31]))
correlation [,1]
Time -0.0123225709
V1 -0.1013472986
V2 0.0912886503
V3 -0.1929608271
V4 0.1334474862
V5 -0.0949742990
V6 -0.0436431607
V7 -0.1872565915
V8 0.0198751239
V9 -0.0977326861
V10 -0.2168829436
V11 0.1548756447
V12 -0.2605929249
V13 -0.0045697788
V14 -0.3025436958
V15 -0.0042234023
V16 -0.1965389403
V17 -0.3264810672
V18 -0.1114852539
V19 0.0347830130
V20 0.0200903242
V21 0.0404133806
V22 0.0008053175
V23 -0.0026851557
V24 -0.0072209067
V25 0.0033077056
V26 0.0044553975
V27 0.0175797282
V28 0.0095360409
Amount 0.0056317530The columns labeled as v1, v3, v10, v12, v14, v16, v17, and v18 exhibit a negative correlation, indicating a strong inverse relationship. Conversely, the columns v2, v4, and v11 demonstrate a positive correlation, indicating a direct relationship. The remaining columns do not display a significant correlation with the class variable, suggesting that they are not strongly associated with it.
corrplot(correlation, method = "pie", cl.pos='n')
On the Above observation we can conclude that:
Variables with Strong Negative Correlations: V10 (-0.2168829436) V12 (-0.2605929249) V14 (-0.3025436958) V17 (-0.3264810672) Variables with Moderate Negative Correlations: V3 (-0.1929608271) V7 (-0.1872565915) V16 (-0.1965389403) Variables with Moderate Positive Correlations: V4 (0.1334474862)
#removing less correlated variables.
credit_card <- select(credit_card, V3, V4, V7, V10, V12, V14, V16, V17, Class)
credit_cardsummary(credit_card) V3 V4 V7 V10 V12 V14
Min. :-48.3256 Min. :-5.68317 Min. :-43.5572 Min. :-24.58826 Min. :-18.6837 Min. :-19.2143
1st Qu.: -0.8904 1st Qu.:-0.84864 1st Qu.: -0.5541 1st Qu.: -0.53543 1st Qu.: -0.4056 1st Qu.: -0.4256
Median : 0.1799 Median :-0.01985 Median : 0.0401 Median : -0.09292 Median : 0.1400 Median : 0.0506
Mean : 0.0000 Mean : 0.00000 Mean : 0.0000 Mean : 0.00000 Mean : 0.0000 Mean : 0.0000
3rd Qu.: 1.0272 3rd Qu.: 0.74334 3rd Qu.: 0.5704 3rd Qu.: 0.45392 3rd Qu.: 0.6182 3rd Qu.: 0.4931
Max. : 9.3826 Max. :16.87534 Max. :120.5895 Max. : 23.74514 Max. : 7.8484 Max. : 10.5268
V16 V17 Class
Min. :-14.12985 Min. :-25.16280 0:284315
1st Qu.: -0.46804 1st Qu.: -0.48375 1: 492
Median : 0.06641 Median : -0.06568
Mean : 0.00000 Mean : 0.00000
3rd Qu.: 0.52330 3rd Qu.: 0.39968
Max. : 17.31511 Max. : 9.25353 #Box Plot, Density plot for all Feature vs Class
boxplot_plots <- list()
density_plots <- list()
for (var in c("V3", "V4", "V7", "V10", "V12", "V14", "V16", "V17")) {
boxplot_plot <- ggplot(credit_card, aes(x = factor(Class), y = .data[[var]], fill = factor(Class))) +
geom_boxplot() +
labs(title = paste("Boxplot of", var, "by Class")) +
theme_minimal()
boxplot_plots <- append(boxplot_plots, list(boxplot_plot))
density_plot <- ggplot(credit_card, aes(x = .data[[var]], fill = factor(Class))) +
geom_density(alpha = 0.7) +
labs(title = paste("Density Plot of", var, "by Class")) +
theme_minimal()
density_plots <- append(density_plots, list(density_plot))
}
for (plot in boxplot_plots) {
print(plot)
}
for (plot in density_plots) {
print(plot)
}
#t-test
for (var in c("V3", "V4", "V7", "V10", "V12", "V14", "V16", "V17")) {
class_0 <- credit_card[[var]][credit_card$Class == 0]
class_1 <- credit_card[[var]][credit_card$Class == 1]
t_test <- t.test(class_0, class_1)
cat(paste("Variable:", var, "\n"))
cat(paste("t-test p-value:", t_test$p.value, "\n\n"))
}Variable: V3
t-test p-value: 4.78608140742888e-75
Variable: V4
t-test p-value: 4.58731692138804e-136
Variable: V7
t-test p-value: 4.29223131944152e-52
Variable: V10
t-test p-value: 3.38288365061388e-93
Variable: V12
t-test p-value: 1.48131650283816e-112
Variable: V14
t-test p-value: 1.0401507099026e-140
Variable: V16
t-test p-value: 7.90532563943685e-84
Variable: V17
t-test p-value: 1.55809380578438e-71 #Under Sampling and Over Sampling of data
credit_card_0 <- credit_card[credit_card$Class == 0, ]
credit_card_1 <- credit_card[credit_card$Class == 1, ]
#Undersampling
credit_card_0_down <- credit_card_0[sample(nrow(credit_card_0), 50000), ]
# Oversampling
credit_card_1_over <- credit_card_1[sample(nrow(credit_card_1), 50000, replace = TRUE), ]
#Combining
credit_card <- rbind(credit_card_0_down, credit_card_1_over)
credit_carddim(credit_card)[1] 100000 9summary(credit_card) V3 V4 V7 V10 V12 V14
Min. :-33.6810 Min. :-5.6006 Min. :-43.5572 Min. :-24.58826 Min. :-18.6837 Min. :-19.2143
1st Qu.: -5.1263 1st Qu.:-0.1086 1st Qu.: -3.1046 1st Qu.: -4.61451 1st Qu.: -5.5837 1st Qu.: -6.7633
Median : -1.3711 Median : 1.3096 Median : -0.6451 Median : -0.86790 Median : -0.8357 Median : -1.0225
Mean : -3.5298 Mean : 2.2596 Mean : -2.7760 Mean : -2.82971 Mean : -3.1292 Mean : -3.4857
3rd Qu.: 0.3164 3rd Qu.: 4.2332 3rd Qu.: 0.2716 3rd Qu.: 0.02872 3rd Qu.: 0.2048 3rd Qu.: 0.1219
Max. : 4.1017 Max. :12.6728 Max. : 44.0545 Max. : 15.24569 Max. : 4.8465 Max. : 7.6677
V16 V17 Class
Min. :-14.1299 Min. :-25.1628 0:50000
1st Qu.: -3.5632 1st Qu.: -5.3565 1:50000
Median : -0.6422 Median : -0.5275
Mean : -2.0729 Mean : -3.3316
3rd Qu.: 0.3001 3rd Qu.: 0.2595
Max. : 7.0591 Max. : 7.8934 #Radomizing the data before spliting
credit_card <- credit_card[sample(nrow(credit_card)), ]
credit_card#Spliting the data into training and testing sets
levels(credit_card$Class)=list(No ="0", Yes="1")
set.seed(123)
train_indices <- createDataPartition(credit_card$Class, p = 0.7, list = FALSE)
train_data <- credit_card[train_indices, ]
test_data <- credit_card[-train_indices, ]
# Check the dimensions of the train and test sets
dim(train_data)[1] 70000 9dim(test_data)[1] 30000 9str(train_data)'data.frame': 70000 obs. of 9 variables:
$ V3 : num -9.427 0.71 -0.304 -8.533 -12.788 ...
$ V4 : num 6.58 1.27 2.76 8.34 8.79 ...
$ V7 : num -10.8941 -0.605 -0.0428 -10.9892 -14.2488 ...
$ V10 : num -12.982 -0.273 -0.158 -11.436 -13.074 ...
$ V12 : num -14.154 1.78 -0.936 -14.296 -12.375 ...
$ V14 : num -16.338 -0.838 -1.633 -15.445 -11.471 ...
$ V16 : num -12.375 -0.605 1.426 -12.391 -10.974 ...
$ V17 : num -21.0906 -0.0303 1.5935 -22.5417 -21.0476 ...
$ Class: Factor w/ 2 levels "No","Yes": 2 1 2 2 2 2 1 1 2 2 ...str(test_data)'data.frame': 30000 obs. of 9 variables:
$ V3 : num 1.2821 -3.7318 -3.733 -17.8801 -0.0958 ...
$ V4 : num -1.158 6.679 1.415 9.249 -0.559 ...
$ V7 : num -0.0854 -8.7481 -0.3138 -18.0147 0.6889 ...
$ V10 : num -0.923 -5.666 -1.331 -14.557 -0.647 ...
$ V12 : num 0.381 -6.147 0.299 -10.38 0.432 ...
$ V14 : num 0.497 -4.329 -2.585 -6.918 0.367 ...
$ V16 : num 0.5201 -1.839 1.5142 -10.3282 0.0338 ...
$ V17 : num -0.97 -3.35 2.82 -20.25 -0.28 ...
$ Class: Factor w/ 2 levels "No","Yes": 1 2 1 2 1 1 2 1 2 2 ...Decision tree using c50.
library(C50)
model_boosted <- C5.0(Class ~ ., data = train_data, trials = 30)
pred_boosted <- predict(model_boosted, newdata = test_data, type = "class")
confusion_matrix_boosted <- table(test_data$Class, pred_boosted)
confusion_matrix_boosted pred_boosted
No Yes
No 14993 7
Yes 0 15000accuracy_boosted <- sum(diag(confusion_matrix_boosted)) / sum(confusion_matrix_boosted)
total_error_boosted <- 1 - accuracy_boosted
precision_boosted <- diag(confusion_matrix_boosted) / colSums(confusion_matrix_boosted)
recall_boosted <- diag(confusion_matrix_boosted) / rowSums(confusion_matrix_boosted)
precision_recall_boosted <- data.frame(Precision = precision_boosted, Recall = recall_boosted)
precision_recall_total_boosted <- rbind(precision_recall_boosted, c(mean(precision_boosted), mean(recall_boosted)))
precision_recall_total_boosted$Labels <- c(levels(test_data$Class), "Average")
print(confusion_matrix_boosted) pred_boosted
No Yes
No 14993 7
Yes 0 15000print(accuracy_boosted)[1] 0.9997667print(paste0("Total Error: ", round(total_error_boosted, 4)))[1] "Total Error: 2e-04"print(precision_recall_total_boosted)#Desicion Tree using rpart
credit_data_model <- train(Class ~ ., data = train_data, method = "rpart", trControl = trainControl(method = "cv", number = 5))
credit_data_pred <- predict(credit_data_model, test_data)
credit_data_cm <- confusionMatrix(credit_data_pred, test_data$Class)
credit_data_auc <- credit_data_cm$overall[1] *100
credit_data_aucAccuracy
93.51 credit_data_acc <- credit_data_cm$overall[4] *100
credit_data_accAccuracyUpper
93.78621 cat("Accuracy of Decision Tree Model" ,credit_data_acc,". Summary of ROC Curve ",credit_data_auc)Accuracy of Decision Tree Model 93.78621 . Summary of ROC Curve 93.51credit_data_modelCART
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (5 fold)
Summary of sample sizes: 56000, 56000, 56000, 56000, 56000
Resampling results across tuning parameters:
cp Accuracy Kappa
0.006857143 0.9350143 0.8700286
0.012928571 0.9256143 0.8512286
0.842342857 0.7518143 0.5036286
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was cp = 0.006857143.#Knn
set.seed(1)
knn_credit_card<- train(Class ~ ., data = train_data, trControl = trainControl(method="cv", number=10, classProbs = TRUE, summaryFunction = twoClassSummary) ,method = "knn",metric ="ROC")
knn_credit_cardk-Nearest Neighbors
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
k ROC Sens Spec
5 0.9991857 0.9920571 1
7 0.9991857 0.9896000 1
9 0.9991857 0.9869429 1
ROC was used to select the optimal model using the largest value.
The final value used for the model was k = 9.knn_predictions_prob=predict(knn_credit_card, test_data, type="prob")
knn_predictions= prediction(knn_predictions_prob$Yes,test_data$Class)
performance(knn_predictions, measure = "auc")A performance instance
'Area under the ROC curve'knn_predicted_labels = predict(knn_credit_card, test_data)
confusionMatrix(knn_predicted_labels, test_data$Class ,positive="Yes", mode="everything")Confusion Matrix and Statistics
Reference
Prediction No Yes
No 14808 0
Yes 192 15000
Accuracy : 0.9936
95% CI : (0.9926, 0.9945)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.9872
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 1.0000
Specificity : 0.9872
Pos Pred Value : 0.9874
Neg Pred Value : 1.0000
Precision : 0.9874
Recall : 1.0000
F1 : 0.9936
Prevalence : 0.5000
Detection Rate : 0.5000
Detection Prevalence : 0.5064
Balanced Accuracy : 0.9936
'Positive' Class : Yes
plot(knn_predictions_prob)
Accuracy : 0.9938
set.seed(1)
knn_smote<- train(Class ~ ., data = train_data,trControl =trainControl(method="cv", number=10, classProbs = TRUE, summaryFunction = twoClassSummary,sampling="smote") ,method = "knn",metric ="ROC")Loading required package: recipes
Attaching package: ‘recipes’
The following object is masked from ‘package:stringr’:
fixed
The following object is masked from ‘package:stats’:
stepknn_smotek-Nearest Neighbors
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Addtional sampling using SMOTE
Resampling results across tuning parameters:
k ROC Sens Spec
5 0.9991857 0.9920571 1
7 0.9991857 0.9896000 1
9 0.9991857 0.9869429 1
ROC was used to select the optimal model using the largest value.
The final value used for the model was k = 9.knn_smote_predictions_prob=predict(knn_smote, test_data, type="prob")
knn_smote_predictions= prediction(knn_smote_predictions_prob$Yes,test_data$Class)
performance(knn_smote_predictions, measure = "auc")A performance instance
'Area under the ROC curve'knn_smote_predicted_labels = predict(knn_smote, test_data)
confusionMatrix(knn_smote_predicted_labels, test_data$Class ,positive="Yes", mode="everything")Confusion Matrix and Statistics
Reference
Prediction No Yes
No 14808 0
Yes 192 15000
Accuracy : 0.9936
95% CI : (0.9926, 0.9945)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.9872
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 1.0000
Specificity : 0.9872
Pos Pred Value : 0.9874
Neg Pred Value : 1.0000
Precision : 0.9874
Recall : 1.0000
F1 : 0.9936
Prevalence : 0.5000
Detection Rate : 0.5000
Detection Prevalence : 0.5064
Balanced Accuracy : 0.9936
'Positive' Class : Yes
Accuracy : 0.9938
#SVM
set.seed(1)
svm_credit<- train(Class ~ ., data = train_data,
trControl = trainControl(method="cv", number=10, classProbs = TRUE, summaryFunction = twoClassSummary) ,method = "svmLinear",metric ="ROC")
svm_creditSupport Vector Machines with Linear Kernel
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results:
ROC Sens Spec
0.9791728 0.9754857 0.9095714
Tuning parameter 'C' was held constant at a value of 1svm_predictions_prob=predict(svm_credit, test_data, type="prob")
svm_predictions= prediction(svm_predictions_prob$Yes,test_data$Class)
svm_predictions_prob
performance(svm_predictions, measure = "auc")A performance instance
'Area under the ROC curve'svm_predicted_labels = predict(svm_credit, test_data)
confusionMatrix(svm_predicted_labels, test_data$Class ,positive="Yes", mode="everything")Confusion Matrix and Statistics
Reference
Prediction No Yes
No 14594 1329
Yes 406 13671
Accuracy : 0.9422
95% CI : (0.9395, 0.9448)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.8843
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 0.9114
Specificity : 0.9729
Pos Pred Value : 0.9712
Neg Pred Value : 0.9165
Precision : 0.9712
Recall : 0.9114
F1 : 0.9403
Prevalence : 0.5000
Detection Rate : 0.4557
Detection Prevalence : 0.4692
Balanced Accuracy : 0.9422
'Positive' Class : Yes
Accuracy : 0.9421
#SVM Smote
set.seed(1)
ctrl=trainControl(method="cv", number=10, classProbs = TRUE, summaryFunction = twoClassSummary,
sampling="smote")
svm_smote<-train(Class ~ ., data = train_data, method = "svmLinear", verbose=FALSE, metric ="ROC",
trControl= ctrl)
svm_smoteSupport Vector Machines with Linear Kernel
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Addtional sampling using SMOTE
Resampling results:
ROC Sens Spec
0.9791728 0.9754857 0.9095714
Tuning parameter 'C' was held constant at a value of 1svm_smote_predictions_prob=predict(svm_smote, test_data, type="prob")
svm_smote_predictions= prediction(svm_smote_predictions_prob$Yes,test_data$Class)
performance(svm_smote_predictions, measure = "auc")A performance instance
'Area under the ROC curve'svm_smote_predicted_labels = predict(svm_smote, test_data)
confusionMatrix(svm_smote_predicted_labels, test_data$Class ,positive="Yes", mode="everything")Confusion Matrix and Statistics
Reference
Prediction No Yes
No 14594 1329
Yes 406 13671
Accuracy : 0.9422
95% CI : (0.9395, 0.9448)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.8843
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 0.9114
Specificity : 0.9729
Pos Pred Value : 0.9712
Neg Pred Value : 0.9165
Precision : 0.9712
Recall : 0.9114
F1 : 0.9403
Prevalence : 0.5000
Detection Rate : 0.4557
Detection Prevalence : 0.4692
Balanced Accuracy : 0.9422
'Positive' Class : Yes
Accuracy : 0.9421
SVM Radial
set.seed(1)
credit_rsvm_model <- train(Class ~ ., data = train_data, method = "svmRadial", trControl = trainControl(method = "cv", number = 10))
credit_rsvm_modelSupport Vector Machines with Radial Basis Function Kernel
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
C Accuracy Kappa
0.25 0.9572714 0.9145429
0.50 0.9648571 0.9297143
1.00 0.9703429 0.9406857
Tuning parameter 'sigma' was held constant at a value of 1.044916
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were sigma = 1.044916 and C = 1.credit_rsvm_pred <- predict(credit_rsvm_model, test_data)
credit_rsvm_cm <- confusionMatrix(credit_rsvm_pred, test_data$Class)
credit_rsvm_auc <- credit_rsvm_cm$overall[1] *100
credit_rsvm_acc <- credit_rsvm_cm$overall[4] *100
credit_rsvm_aucAccuracy
96.98333 Accuracy 96.85
credit_rsvm_modelSupport Vector Machines with Radial Basis Function Kernel
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
C Accuracy Kappa
0.25 0.9572714 0.9145429
0.50 0.9648571 0.9297143
1.00 0.9703429 0.9406857
Tuning parameter 'sigma' was held constant at a value of 1.044916
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were sigma = 1.044916 and C = 1.credit_rsvm_cmConfusion Matrix and Statistics
Reference
Prediction No Yes
No 14834 739
Yes 166 14261
Accuracy : 0.9698
95% CI : (0.9678, 0.9717)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.9397
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 0.9889
Specificity : 0.9507
Pos Pred Value : 0.9525
Neg Pred Value : 0.9885
Prevalence : 0.5000
Detection Rate : 0.4945
Detection Prevalence : 0.5191
Balanced Accuracy : 0.9698
'Positive' Class : No
#RandomForest
set.seed(1)
credit_rf_model <- train(Class ~ ., data = train_data, method = "rf",importance= T,metric = "ROC", trControl = trainControl(method = "cv", number = 10, classProbs = TRUE))Warning: The metric "ROC" was not in the result set. Accuracy will be used instead.credit_rf_modelRandom Forest
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
mtry Accuracy Kappa
2 0.9998286 0.9996571
5 0.9997000 0.9994000
8 0.9989000 0.9978000
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.credit_rf_pred <- predict(credit_rf_model, test_data)
credit_rf_cm <- confusionMatrix(credit_rf_pred, test_data$Class)
credit_rf_auc <- credit_rf_cm$overall[1]
credit_rf_acc <- credit_rf_cm$overall[4]
credit_rf_aucAccuracy
0.9998 varImp(credit_rf_model)rf variable importancecredit_rf_modelRandom Forest
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
mtry Accuracy Kappa
2 0.9998286 0.9996571
5 0.9997000 0.9994000
8 0.9989000 0.9978000
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.credit_rf_cmConfusion Matrix and Statistics
Reference
Prediction No Yes
No 14994 0
Yes 6 15000
Accuracy : 0.9998
95% CI : (0.9996, 0.9999)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2e-16
Kappa : 0.9996
Mcnemar's Test P-Value : 0.04123
Sensitivity : 0.9996
Specificity : 1.0000
Pos Pred Value : 1.0000
Neg Pred Value : 0.9996
Prevalence : 0.5000
Detection Rate : 0.4998
Detection Prevalence : 0.4998
Balanced Accuracy : 0.9998
'Positive' Class : No
#Gradient Boosted Tree model
set.seed(1)
credit_gbm_model <- train(Class ~ ., data = train_data, method = "gbm", trControl = trainControl(method = "cv", number = 10))Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2504 nan 0.1000 0.0680
2 1.1398 nan 0.1000 0.0555
3 1.0474 nan 0.1000 0.0463
4 0.9692 nan 0.1000 0.0391
5 0.9029 nan 0.1000 0.0331
6 0.8464 nan 0.1000 0.0282
7 0.7960 nan 0.1000 0.0251
8 0.7512 nan 0.1000 0.0223
9 0.7128 nan 0.1000 0.0193
10 0.6779 nan 0.1000 0.0174
20 0.4760 nan 0.1000 0.0060
40 0.3520 nan 0.1000 0.0017
60 0.3125 nan 0.1000 0.0008
80 0.2941 nan 0.1000 0.0004
100 0.2830 nan 0.1000 0.0001
120 0.2746 nan 0.1000 0.0002
140 0.2672 nan 0.1000 0.0000
150 0.2647 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2431 nan 0.1000 0.0717
2 1.1259 nan 0.1000 0.0585
3 1.0281 nan 0.1000 0.0489
4 0.9441 nan 0.1000 0.0419
5 0.8748 nan 0.1000 0.0346
6 0.8143 nan 0.1000 0.0300
7 0.7589 nan 0.1000 0.0277
8 0.7116 nan 0.1000 0.0235
9 0.6699 nan 0.1000 0.0210
10 0.6339 nan 0.1000 0.0180
20 0.4280 nan 0.1000 0.0069
40 0.2994 nan 0.1000 0.0015
60 0.2631 nan 0.1000 0.0006
80 0.2420 nan 0.1000 0.0002
100 0.2275 nan 0.1000 0.0002
120 0.2140 nan 0.1000 0.0003
140 0.2008 nan 0.1000 0.0002
150 0.1953 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2401 nan 0.1000 0.0732
2 1.1207 nan 0.1000 0.0600
3 1.0206 nan 0.1000 0.0498
4 0.9366 nan 0.1000 0.0420
5 0.8628 nan 0.1000 0.0367
6 0.7974 nan 0.1000 0.0327
7 0.7434 nan 0.1000 0.0272
8 0.6943 nan 0.1000 0.0245
9 0.6531 nan 0.1000 0.0205
10 0.6153 nan 0.1000 0.0187
20 0.3995 nan 0.1000 0.0069
40 0.2742 nan 0.1000 0.0012
60 0.2335 nan 0.1000 0.0003
80 0.2093 nan 0.1000 0.0005
100 0.1892 nan 0.1000 0.0003
120 0.1747 nan 0.1000 0.0002
140 0.1591 nan 0.1000 0.0002
150 0.1534 nan 0.1000 0.0003
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2501 nan 0.1000 0.0681
2 1.1389 nan 0.1000 0.0555
3 1.0459 nan 0.1000 0.0462
4 0.9677 nan 0.1000 0.0390
5 0.9016 nan 0.1000 0.0330
6 0.8444 nan 0.1000 0.0283
7 0.7943 nan 0.1000 0.0252
8 0.7504 nan 0.1000 0.0219
9 0.7121 nan 0.1000 0.0192
10 0.6782 nan 0.1000 0.0169
20 0.4747 nan 0.1000 0.0063
40 0.3528 nan 0.1000 0.0010
60 0.3128 nan 0.1000 0.0006
80 0.2937 nan 0.1000 0.0006
100 0.2820 nan 0.1000 0.0002
120 0.2739 nan 0.1000 0.0001
140 0.2681 nan 0.1000 0.0001
150 0.2657 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2427 nan 0.1000 0.0717
2 1.1254 nan 0.1000 0.0585
3 1.0259 nan 0.1000 0.0497
4 0.9445 nan 0.1000 0.0406
5 0.8722 nan 0.1000 0.0360
6 0.8105 nan 0.1000 0.0307
7 0.7580 nan 0.1000 0.0262
8 0.7136 nan 0.1000 0.0221
9 0.6708 nan 0.1000 0.0214
10 0.6337 nan 0.1000 0.0184
20 0.4229 nan 0.1000 0.0057
40 0.2992 nan 0.1000 0.0012
60 0.2614 nan 0.1000 0.0008
80 0.2408 nan 0.1000 0.0005
100 0.2254 nan 0.1000 0.0003
120 0.2124 nan 0.1000 0.0003
140 0.2002 nan 0.1000 0.0000
150 0.1947 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2400 nan 0.1000 0.0731
2 1.1198 nan 0.1000 0.0599
3 1.0199 nan 0.1000 0.0497
4 0.9354 nan 0.1000 0.0422
5 0.8618 nan 0.1000 0.0367
6 0.7993 nan 0.1000 0.0313
7 0.7465 nan 0.1000 0.0263
8 0.6958 nan 0.1000 0.0254
9 0.6527 nan 0.1000 0.0216
10 0.6150 nan 0.1000 0.0186
20 0.3974 nan 0.1000 0.0071
40 0.2729 nan 0.1000 0.0012
60 0.2303 nan 0.1000 0.0004
80 0.2043 nan 0.1000 0.0004
100 0.1861 nan 0.1000 0.0001
120 0.1696 nan 0.1000 0.0002
140 0.1570 nan 0.1000 0.0004
150 0.1513 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2495 nan 0.1000 0.0681
2 1.1385 nan 0.1000 0.0557
3 1.0456 nan 0.1000 0.0464
4 0.9676 nan 0.1000 0.0391
5 0.9012 nan 0.1000 0.0332
6 0.8448 nan 0.1000 0.0283
7 0.7945 nan 0.1000 0.0252
8 0.7498 nan 0.1000 0.0223
9 0.7111 nan 0.1000 0.0192
10 0.6765 nan 0.1000 0.0171
20 0.4745 nan 0.1000 0.0061
40 0.3528 nan 0.1000 0.0011
60 0.3127 nan 0.1000 0.0006
80 0.2945 nan 0.1000 0.0001
100 0.2819 nan 0.1000 0.0003
120 0.2736 nan 0.1000 0.0002
140 0.2670 nan 0.1000 0.0001
150 0.2644 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2426 nan 0.1000 0.0718
2 1.1254 nan 0.1000 0.0586
3 1.0263 nan 0.1000 0.0498
4 0.9444 nan 0.1000 0.0409
5 0.8724 nan 0.1000 0.0361
6 0.8124 nan 0.1000 0.0301
7 0.7570 nan 0.1000 0.0276
8 0.7119 nan 0.1000 0.0226
9 0.6693 nan 0.1000 0.0211
10 0.6351 nan 0.1000 0.0168
20 0.4228 nan 0.1000 0.0074
40 0.2974 nan 0.1000 0.0016
60 0.2618 nan 0.1000 0.0006
80 0.2432 nan 0.1000 0.0004
100 0.2278 nan 0.1000 0.0002
120 0.2147 nan 0.1000 0.0002
140 0.2003 nan 0.1000 0.0001
150 0.1938 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2398 nan 0.1000 0.0731
2 1.1200 nan 0.1000 0.0602
3 1.0199 nan 0.1000 0.0499
4 0.9319 nan 0.1000 0.0442
5 0.8589 nan 0.1000 0.0364
6 0.7966 nan 0.1000 0.0310
7 0.7426 nan 0.1000 0.0268
8 0.6931 nan 0.1000 0.0246
9 0.6512 nan 0.1000 0.0210
10 0.6138 nan 0.1000 0.0188
20 0.3980 nan 0.1000 0.0062
40 0.2724 nan 0.1000 0.0016
60 0.2293 nan 0.1000 0.0004
80 0.2071 nan 0.1000 0.0004
100 0.1893 nan 0.1000 0.0006
120 0.1732 nan 0.1000 0.0004
140 0.1582 nan 0.1000 0.0003
150 0.1529 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2500 nan 0.1000 0.0681
2 1.1390 nan 0.1000 0.0555
3 1.0467 nan 0.1000 0.0462
4 0.9687 nan 0.1000 0.0390
5 0.9025 nan 0.1000 0.0331
6 0.8457 nan 0.1000 0.0284
7 0.7956 nan 0.1000 0.0253
8 0.7519 nan 0.1000 0.0219
9 0.7130 nan 0.1000 0.0194
10 0.6793 nan 0.1000 0.0168
20 0.4762 nan 0.1000 0.0062
40 0.3541 nan 0.1000 0.0011
60 0.3143 nan 0.1000 0.0006
80 0.2955 nan 0.1000 0.0004
100 0.2840 nan 0.1000 0.0001
120 0.2759 nan 0.1000 0.0001
140 0.2695 nan 0.1000 0.0001
150 0.2669 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2431 nan 0.1000 0.0715
2 1.1259 nan 0.1000 0.0586
3 1.0267 nan 0.1000 0.0495
4 0.9454 nan 0.1000 0.0406
5 0.8752 nan 0.1000 0.0349
6 0.8130 nan 0.1000 0.0311
7 0.7610 nan 0.1000 0.0258
8 0.7122 nan 0.1000 0.0243
9 0.6704 nan 0.1000 0.0208
10 0.6358 nan 0.1000 0.0172
20 0.4269 nan 0.1000 0.0064
40 0.3016 nan 0.1000 0.0013
60 0.2635 nan 0.1000 0.0004
80 0.2422 nan 0.1000 0.0006
100 0.2250 nan 0.1000 0.0003
120 0.2137 nan 0.1000 0.0003
140 0.2010 nan 0.1000 0.0003
150 0.1930 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2397 nan 0.1000 0.0729
2 1.1207 nan 0.1000 0.0599
3 1.0207 nan 0.1000 0.0497
4 0.9329 nan 0.1000 0.0439
5 0.8598 nan 0.1000 0.0367
6 0.7981 nan 0.1000 0.0308
7 0.7411 nan 0.1000 0.0284
8 0.6942 nan 0.1000 0.0234
9 0.6513 nan 0.1000 0.0214
10 0.6161 nan 0.1000 0.0175
20 0.3998 nan 0.1000 0.0072
40 0.2743 nan 0.1000 0.0014
60 0.2305 nan 0.1000 0.0003
80 0.2058 nan 0.1000 0.0005
100 0.1866 nan 0.1000 0.0003
120 0.1717 nan 0.1000 0.0004
140 0.1606 nan 0.1000 0.0005
150 0.1559 nan 0.1000 0.0003
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2504 nan 0.1000 0.0679
2 1.1384 nan 0.1000 0.0555
3 1.0460 nan 0.1000 0.0463
4 0.9681 nan 0.1000 0.0388
5 0.9020 nan 0.1000 0.0331
6 0.8451 nan 0.1000 0.0283
7 0.7947 nan 0.1000 0.0251
8 0.7505 nan 0.1000 0.0220
9 0.7119 nan 0.1000 0.0192
10 0.6770 nan 0.1000 0.0174
20 0.4757 nan 0.1000 0.0059
40 0.3535 nan 0.1000 0.0018
60 0.3136 nan 0.1000 0.0005
80 0.2942 nan 0.1000 0.0005
100 0.2841 nan 0.1000 0.0003
120 0.2748 nan 0.1000 0.0002
140 0.2690 nan 0.1000 0.0001
150 0.2662 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2432 nan 0.1000 0.0716
2 1.1259 nan 0.1000 0.0585
3 1.0267 nan 0.1000 0.0496
4 0.9448 nan 0.1000 0.0410
5 0.8743 nan 0.1000 0.0350
6 0.8148 nan 0.1000 0.0295
7 0.7593 nan 0.1000 0.0275
8 0.7119 nan 0.1000 0.0238
9 0.6728 nan 0.1000 0.0196
10 0.6357 nan 0.1000 0.0185
20 0.4221 nan 0.1000 0.0070
40 0.3016 nan 0.1000 0.0016
60 0.2643 nan 0.1000 0.0004
80 0.2449 nan 0.1000 0.0003
100 0.2276 nan 0.1000 0.0003
120 0.2138 nan 0.1000 0.0003
140 0.2039 nan 0.1000 0.0004
150 0.1991 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2398 nan 0.1000 0.0732
2 1.1199 nan 0.1000 0.0599
3 1.0196 nan 0.1000 0.0498
4 0.9360 nan 0.1000 0.0418
5 0.8622 nan 0.1000 0.0369
6 0.7960 nan 0.1000 0.0329
7 0.7418 nan 0.1000 0.0269
8 0.6948 nan 0.1000 0.0233
9 0.6520 nan 0.1000 0.0214
10 0.6137 nan 0.1000 0.0193
20 0.3980 nan 0.1000 0.0065
40 0.2743 nan 0.1000 0.0017
60 0.2317 nan 0.1000 0.0006
80 0.2064 nan 0.1000 0.0003
100 0.1867 nan 0.1000 0.0006
120 0.1722 nan 0.1000 0.0003
140 0.1581 nan 0.1000 0.0001
150 0.1540 nan 0.1000 0.0003
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2507 nan 0.1000 0.0678
2 1.1406 nan 0.1000 0.0554
3 1.0483 nan 0.1000 0.0463
4 0.9701 nan 0.1000 0.0390
5 0.9038 nan 0.1000 0.0331
6 0.8471 nan 0.1000 0.0282
7 0.7963 nan 0.1000 0.0253
8 0.7524 nan 0.1000 0.0219
9 0.7136 nan 0.1000 0.0193
10 0.6791 nan 0.1000 0.0172
20 0.4775 nan 0.1000 0.0060
40 0.3551 nan 0.1000 0.0011
60 0.3145 nan 0.1000 0.0006
80 0.2953 nan 0.1000 0.0007
100 0.2838 nan 0.1000 0.0004
120 0.2757 nan 0.1000 0.0001
140 0.2695 nan 0.1000 0.0000
150 0.2660 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2429 nan 0.1000 0.0714
2 1.1257 nan 0.1000 0.0586
3 1.0265 nan 0.1000 0.0496
4 0.9440 nan 0.1000 0.0411
5 0.8743 nan 0.1000 0.0347
6 0.8137 nan 0.1000 0.0301
7 0.7588 nan 0.1000 0.0276
8 0.7115 nan 0.1000 0.0235
9 0.6703 nan 0.1000 0.0206
10 0.6330 nan 0.1000 0.0186
20 0.4273 nan 0.1000 0.0060
40 0.3020 nan 0.1000 0.0013
60 0.2630 nan 0.1000 0.0008
80 0.2422 nan 0.1000 0.0004
100 0.2253 nan 0.1000 0.0001
120 0.2124 nan 0.1000 0.0002
140 0.2004 nan 0.1000 0.0002
150 0.1952 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2403 nan 0.1000 0.0730
2 1.1203 nan 0.1000 0.0597
3 1.0204 nan 0.1000 0.0498
4 0.9360 nan 0.1000 0.0421
5 0.8607 nan 0.1000 0.0379
6 0.7993 nan 0.1000 0.0307
7 0.7451 nan 0.1000 0.0270
8 0.6954 nan 0.1000 0.0248
9 0.6523 nan 0.1000 0.0214
10 0.6158 nan 0.1000 0.0182
20 0.4009 nan 0.1000 0.0061
40 0.2760 nan 0.1000 0.0018
60 0.2331 nan 0.1000 0.0007
80 0.2094 nan 0.1000 0.0006
100 0.1897 nan 0.1000 0.0005
120 0.1730 nan 0.1000 0.0002
140 0.1611 nan 0.1000 0.0005
150 0.1552 nan 0.1000 0.0003
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2501 nan 0.1000 0.0681
2 1.1394 nan 0.1000 0.0556
3 1.0468 nan 0.1000 0.0464
4 0.9684 nan 0.1000 0.0392
5 0.9026 nan 0.1000 0.0331
6 0.8458 nan 0.1000 0.0281
7 0.7948 nan 0.1000 0.0253
8 0.7498 nan 0.1000 0.0224
9 0.7114 nan 0.1000 0.0193
10 0.6767 nan 0.1000 0.0173
20 0.4759 nan 0.1000 0.0057
40 0.3520 nan 0.1000 0.0019
60 0.3139 nan 0.1000 0.0006
80 0.2936 nan 0.1000 0.0003
100 0.2834 nan 0.1000 0.0001
120 0.2745 nan 0.1000 0.0002
140 0.2695 nan 0.1000 0.0000
150 0.2653 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2434 nan 0.1000 0.0719
2 1.1260 nan 0.1000 0.0587
3 1.0263 nan 0.1000 0.0498
4 0.9449 nan 0.1000 0.0405
5 0.8731 nan 0.1000 0.0359
6 0.8133 nan 0.1000 0.0300
7 0.7613 nan 0.1000 0.0258
8 0.7131 nan 0.1000 0.0241
9 0.6711 nan 0.1000 0.0208
10 0.6339 nan 0.1000 0.0184
20 0.4265 nan 0.1000 0.0061
40 0.2992 nan 0.1000 0.0010
60 0.2630 nan 0.1000 0.0003
80 0.2432 nan 0.1000 0.0001
100 0.2253 nan 0.1000 0.0003
120 0.2125 nan 0.1000 0.0003
140 0.2003 nan 0.1000 0.0001
150 0.1954 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2396 nan 0.1000 0.0733
2 1.1197 nan 0.1000 0.0599
3 1.0197 nan 0.1000 0.0501
4 0.9349 nan 0.1000 0.0421
5 0.8589 nan 0.1000 0.0379
6 0.7954 nan 0.1000 0.0318
7 0.7414 nan 0.1000 0.0269
8 0.6951 nan 0.1000 0.0230
9 0.6520 nan 0.1000 0.0217
10 0.6130 nan 0.1000 0.0195
20 0.3973 nan 0.1000 0.0072
40 0.2750 nan 0.1000 0.0015
60 0.2283 nan 0.1000 0.0009
80 0.2021 nan 0.1000 0.0009
100 0.1828 nan 0.1000 0.0005
120 0.1719 nan 0.1000 0.0003
140 0.1583 nan 0.1000 0.0002
150 0.1517 nan 0.1000 0.0004
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2503 nan 0.1000 0.0680
2 1.1394 nan 0.1000 0.0554
3 1.0468 nan 0.1000 0.0462
4 0.9690 nan 0.1000 0.0389
5 0.9026 nan 0.1000 0.0329
6 0.8464 nan 0.1000 0.0283
7 0.7956 nan 0.1000 0.0254
8 0.7510 nan 0.1000 0.0222
9 0.7122 nan 0.1000 0.0193
10 0.6772 nan 0.1000 0.0173
20 0.4757 nan 0.1000 0.0061
40 0.3535 nan 0.1000 0.0018
60 0.3147 nan 0.1000 0.0004
80 0.2940 nan 0.1000 0.0002
100 0.2829 nan 0.1000 0.0001
120 0.2745 nan 0.1000 0.0001
140 0.2677 nan 0.1000 0.0002
150 0.2654 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2433 nan 0.1000 0.0715
2 1.1262 nan 0.1000 0.0584
3 1.0271 nan 0.1000 0.0496
4 0.9448 nan 0.1000 0.0409
5 0.8747 nan 0.1000 0.0348
6 0.8125 nan 0.1000 0.0310
7 0.7580 nan 0.1000 0.0275
8 0.7135 nan 0.1000 0.0224
9 0.6717 nan 0.1000 0.0209
10 0.6355 nan 0.1000 0.0180
20 0.4258 nan 0.1000 0.0064
40 0.3000 nan 0.1000 0.0017
60 0.2612 nan 0.1000 0.0007
80 0.2417 nan 0.1000 0.0002
100 0.2244 nan 0.1000 0.0004
120 0.2111 nan 0.1000 0.0002
140 0.1987 nan 0.1000 0.0002
150 0.1935 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2399 nan 0.1000 0.0729
2 1.1207 nan 0.1000 0.0598
3 1.0203 nan 0.1000 0.0502
4 0.9347 nan 0.1000 0.0428
5 0.8614 nan 0.1000 0.0367
6 0.7962 nan 0.1000 0.0326
7 0.7416 nan 0.1000 0.0269
8 0.6926 nan 0.1000 0.0245
9 0.6518 nan 0.1000 0.0204
10 0.6145 nan 0.1000 0.0187
20 0.3981 nan 0.1000 0.0064
40 0.2743 nan 0.1000 0.0016
60 0.2290 nan 0.1000 0.0006
80 0.2044 nan 0.1000 0.0007
100 0.1863 nan 0.1000 0.0004
120 0.1704 nan 0.1000 0.0005
140 0.1582 nan 0.1000 0.0003
150 0.1515 nan 0.1000 0.0003
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2505 nan 0.1000 0.0680
2 1.1395 nan 0.1000 0.0556
3 1.0472 nan 0.1000 0.0461
4 0.9684 nan 0.1000 0.0394
5 0.9017 nan 0.1000 0.0332
6 0.8453 nan 0.1000 0.0282
7 0.7946 nan 0.1000 0.0253
8 0.7497 nan 0.1000 0.0222
9 0.7110 nan 0.1000 0.0193
10 0.6769 nan 0.1000 0.0170
20 0.4759 nan 0.1000 0.0063
40 0.3546 nan 0.1000 0.0011
60 0.3153 nan 0.1000 0.0006
80 0.2953 nan 0.1000 0.0005
100 0.2843 nan 0.1000 0.0001
120 0.2756 nan 0.1000 0.0001
140 0.2683 nan 0.1000 0.0002
150 0.2653 nan 0.1000 0.0000
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2429 nan 0.1000 0.0716
2 1.1253 nan 0.1000 0.0585
3 1.0261 nan 0.1000 0.0497
4 0.9446 nan 0.1000 0.0407
5 0.8729 nan 0.1000 0.0359
6 0.8133 nan 0.1000 0.0297
7 0.7581 nan 0.1000 0.0275
8 0.7111 nan 0.1000 0.0235
9 0.6698 nan 0.1000 0.0206
10 0.6317 nan 0.1000 0.0189
20 0.4229 nan 0.1000 0.0070
40 0.2990 nan 0.1000 0.0016
60 0.2623 nan 0.1000 0.0004
80 0.2429 nan 0.1000 0.0005
100 0.2242 nan 0.1000 0.0006
120 0.2109 nan 0.1000 0.0004
140 0.2020 nan 0.1000 0.0002
150 0.1939 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2401 nan 0.1000 0.0732
2 1.1200 nan 0.1000 0.0600
3 1.0198 nan 0.1000 0.0499
4 0.9354 nan 0.1000 0.0421
5 0.8591 nan 0.1000 0.0379
6 0.7969 nan 0.1000 0.0311
7 0.7420 nan 0.1000 0.0274
8 0.6926 nan 0.1000 0.0247
9 0.6519 nan 0.1000 0.0203
10 0.6163 nan 0.1000 0.0176
20 0.3984 nan 0.1000 0.0072
40 0.2769 nan 0.1000 0.0018
60 0.2309 nan 0.1000 0.0008
80 0.2057 nan 0.1000 0.0008
100 0.1864 nan 0.1000 0.0004
120 0.1711 nan 0.1000 0.0003
140 0.1577 nan 0.1000 0.0003
150 0.1513 nan 0.1000 0.0002
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2504 nan 0.1000 0.0682
2 1.1391 nan 0.1000 0.0557
3 1.0462 nan 0.1000 0.0465
4 0.9678 nan 0.1000 0.0391
5 0.9018 nan 0.1000 0.0328
6 0.8449 nan 0.1000 0.0283
7 0.7937 nan 0.1000 0.0256
8 0.7490 nan 0.1000 0.0223
9 0.7101 nan 0.1000 0.0193
10 0.6756 nan 0.1000 0.0170
20 0.4751 nan 0.1000 0.0061
40 0.3511 nan 0.1000 0.0019
60 0.3122 nan 0.1000 0.0009
80 0.2939 nan 0.1000 0.0006
100 0.2816 nan 0.1000 0.0003
120 0.2731 nan 0.1000 0.0002
140 0.2662 nan 0.1000 0.0001
150 0.2636 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2424 nan 0.1000 0.0717
2 1.1246 nan 0.1000 0.0587
3 1.0268 nan 0.1000 0.0488
4 0.9429 nan 0.1000 0.0419
5 0.8731 nan 0.1000 0.0347
6 0.8109 nan 0.1000 0.0309
7 0.7584 nan 0.1000 0.0261
8 0.7100 nan 0.1000 0.0241
9 0.6706 nan 0.1000 0.0198
10 0.6340 nan 0.1000 0.0184
20 0.4254 nan 0.1000 0.0078
40 0.2991 nan 0.1000 0.0011
60 0.2612 nan 0.1000 0.0004
80 0.2418 nan 0.1000 0.0004
100 0.2255 nan 0.1000 0.0005
120 0.2128 nan 0.1000 0.0002
140 0.1999 nan 0.1000 0.0002
150 0.1943 nan 0.1000 0.0001
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2398 nan 0.1000 0.0735
2 1.1199 nan 0.1000 0.0599
3 1.0195 nan 0.1000 0.0501
4 0.9332 nan 0.1000 0.0431
5 0.8608 nan 0.1000 0.0361
6 0.7951 nan 0.1000 0.0328
7 0.7383 nan 0.1000 0.0284
8 0.6895 nan 0.1000 0.0245
9 0.6478 nan 0.1000 0.0208
10 0.6120 nan 0.1000 0.0179
20 0.3959 nan 0.1000 0.0070
40 0.2699 nan 0.1000 0.0014
60 0.2279 nan 0.1000 0.0006
80 0.2024 nan 0.1000 0.0007
100 0.1855 nan 0.1000 0.0002
120 0.1706 nan 0.1000 0.0009
140 0.1558 nan 0.1000 0.0003
150 0.1489 nan 0.1000 0.0003
Iter TrainDeviance ValidDeviance StepSize Improve
1 1.2399 nan 0.1000 0.0732
2 1.1198 nan 0.1000 0.0600
3 1.0201 nan 0.1000 0.0496
4 0.9322 nan 0.1000 0.0441
5 0.8601 nan 0.1000 0.0360
6 0.7966 nan 0.1000 0.0316
7 0.7434 nan 0.1000 0.0267
8 0.6959 nan 0.1000 0.0236
9 0.6526 nan 0.1000 0.0216
10 0.6134 nan 0.1000 0.0195
20 0.3973 nan 0.1000 0.0067
40 0.2727 nan 0.1000 0.0019
60 0.2271 nan 0.1000 0.0007
80 0.2037 nan 0.1000 0.0009
100 0.1866 nan 0.1000 0.0002
120 0.1717 nan 0.1000 0.0004
140 0.1587 nan 0.1000 0.0008
150 0.1509 nan 0.1000 0.0003credit_gbm_predictions <- predict(credit_gbm_model,test_data)
credit_gbm_cm <- confusionMatrix(credit_gbm_predictions, test_data$Class)
credit_gbm_auc <- credit_gbm_cm$overall[1]
credit_gbm_aucAccuracy
0.9745 credit_gbm_acc <- credit_gbm_cm$overall[4]
credit_gbm_accAccuracyUpper
0.9762544 Accuracy : 0.9698
credit_gbm_modelStochastic Gradient Boosting
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
interaction.depth n.trees Accuracy Kappa
1 50 0.9426857 0.8853714
1 100 0.9458286 0.8916571
1 150 0.9481714 0.8963429
2 50 0.9484000 0.8968000
2 100 0.9568143 0.9136286
2 150 0.9626000 0.9252000
3 50 0.9553571 0.9107143
3 100 0.9643714 0.9287429
3 150 0.9719429 0.9438857
Tuning parameter 'shrinkage' was held constant at a value of 0.1
Tuning parameter 'n.minobsinnode' was held
constant at a value of 10
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were n.trees = 150, interaction.depth = 3, shrinkage = 0.1 and n.minobsinnode
= 10.credit_gbm_cmConfusion Matrix and Statistics
Reference
Prediction No Yes
No 14775 540
Yes 225 14460
Accuracy : 0.9745
95% CI : (0.9727, 0.9763)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.949
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 0.9850
Specificity : 0.9640
Pos Pred Value : 0.9647
Neg Pred Value : 0.9847
Prevalence : 0.5000
Detection Rate : 0.4925
Detection Prevalence : 0.5105
Balanced Accuracy : 0.9745
'Positive' Class : No
XGBoost
set.seed(1)
credit_xgb_model <- train(Class ~ ., data = train_data, method = "xgbTree", trControl = trainControl(method = "cv", number = 10),verbosity = 0)
credit_xgb_modeleXtreme Gradient Boosting
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
eta max_depth colsample_bytree subsample nrounds Accuracy Kappa
0.3 1 0.6 0.50 50 0.9478286 0.8956571
0.3 1 0.6 0.50 100 0.9522429 0.9044857
0.3 1 0.6 0.50 150 0.9573714 0.9147429
0.3 1 0.6 0.75 50 0.9480286 0.8960571
0.3 1 0.6 0.75 100 0.9518286 0.9036571
0.3 1 0.6 0.75 150 0.9557143 0.9114286
0.3 1 0.6 1.00 50 0.9476714 0.8953429
0.3 1 0.6 1.00 100 0.9516429 0.9032857
0.3 1 0.6 1.00 150 0.9554000 0.9108000
0.3 1 0.8 0.50 50 0.9477714 0.8955429
0.3 1 0.8 0.50 100 0.9519429 0.9038857
0.3 1 0.8 0.50 150 0.9569714 0.9139429
0.3 1 0.8 0.75 50 0.9483000 0.8966000
0.3 1 0.8 0.75 100 0.9521571 0.9043143
0.3 1 0.8 0.75 150 0.9563000 0.9126000
0.3 1 0.8 1.00 50 0.9476714 0.8953429
0.3 1 0.8 1.00 100 0.9525714 0.9051429
0.3 1 0.8 1.00 150 0.9552429 0.9104857
0.3 2 0.6 0.50 50 0.9679571 0.9359143
0.3 2 0.6 0.50 100 0.9910429 0.9820857
0.3 2 0.6 0.50 150 0.9963429 0.9926857
0.3 2 0.6 0.75 50 0.9694714 0.9389429
0.3 2 0.6 0.75 100 0.9912286 0.9824571
0.3 2 0.6 0.75 150 0.9964429 0.9928857
0.3 2 0.6 1.00 50 0.9695429 0.9390857
0.3 2 0.6 1.00 100 0.9920286 0.9840571
0.3 2 0.6 1.00 150 0.9964143 0.9928286
0.3 2 0.8 0.50 50 0.9711143 0.9422286
0.3 2 0.8 0.50 100 0.9924429 0.9848857
0.3 2 0.8 0.50 150 0.9967571 0.9935143
0.3 2 0.8 0.75 50 0.9725143 0.9450286
0.3 2 0.8 0.75 100 0.9926429 0.9852857
0.3 2 0.8 0.75 150 0.9965429 0.9930857
0.3 2 0.8 1.00 50 0.9710857 0.9421714
0.3 2 0.8 1.00 100 0.9924714 0.9849429
0.3 2 0.8 1.00 150 0.9965714 0.9931429
0.3 3 0.6 0.50 50 0.9933714 0.9867429
0.3 3 0.6 0.50 100 0.9979571 0.9959143
0.3 3 0.6 0.50 150 0.9987857 0.9975714
0.3 3 0.6 0.75 50 0.9936143 0.9872286
0.3 3 0.6 0.75 100 0.9977857 0.9955714
0.3 3 0.6 0.75 150 0.9988429 0.9976857
0.3 3 0.6 1.00 50 0.9939857 0.9879714
0.3 3 0.6 1.00 100 0.9979857 0.9959714
0.3 3 0.6 1.00 150 0.9990571 0.9981143
0.3 3 0.8 0.50 50 0.9945286 0.9890571
0.3 3 0.8 0.50 100 0.9980143 0.9960286
0.3 3 0.8 0.50 150 0.9990571 0.9981143
0.3 3 0.8 0.75 50 0.9943714 0.9887429
0.3 3 0.8 0.75 100 0.9983143 0.9966286
0.3 3 0.8 0.75 150 0.9990286 0.9980571
0.3 3 0.8 1.00 50 0.9940571 0.9881143
0.3 3 0.8 1.00 100 0.9981000 0.9962000
0.3 3 0.8 1.00 150 0.9989571 0.9979143
0.4 1 0.6 0.50 50 0.9500571 0.9001143
0.4 1 0.6 0.50 100 0.9559571 0.9119143
0.4 1 0.6 0.50 150 0.9603857 0.9207714
0.4 1 0.6 0.75 50 0.9496000 0.8992000
0.4 1 0.6 0.75 100 0.9551571 0.9103143
0.4 1 0.6 0.75 150 0.9605714 0.9211429
0.4 1 0.6 1.00 50 0.9492000 0.8984000
0.4 1 0.6 1.00 100 0.9538143 0.9076286
0.4 1 0.6 1.00 150 0.9598429 0.9196857
0.4 1 0.8 0.50 50 0.9501714 0.9003429
0.4 1 0.8 0.50 100 0.9560143 0.9120286
0.4 1 0.8 0.50 150 0.9613571 0.9227143
0.4 1 0.8 0.75 50 0.9504000 0.9008000
0.4 1 0.8 0.75 100 0.9561286 0.9122571
0.4 1 0.8 0.75 150 0.9596143 0.9192286
0.4 1 0.8 1.00 50 0.9490857 0.8981714
0.4 1 0.8 1.00 100 0.9542714 0.9085429
0.4 1 0.8 1.00 150 0.9597286 0.9194571
0.4 2 0.6 0.50 50 0.9801857 0.9603714
0.4 2 0.6 0.50 100 0.9955857 0.9911714
0.4 2 0.6 0.50 150 0.9974857 0.9949714
0.4 2 0.6 0.75 50 0.9816286 0.9632571
0.4 2 0.6 0.75 100 0.9956000 0.9912000
0.4 2 0.6 0.75 150 0.9974286 0.9948571
0.4 2 0.6 1.00 50 0.9815143 0.9630286
0.4 2 0.6 1.00 100 0.9956000 0.9912000
0.4 2 0.6 1.00 150 0.9975714 0.9951429
0.4 2 0.8 0.50 50 0.9822714 0.9645429
0.4 2 0.8 0.50 100 0.9955857 0.9911714
0.4 2 0.8 0.50 150 0.9973857 0.9947714
0.4 2 0.8 0.75 50 0.9811286 0.9622571
0.4 2 0.8 0.75 100 0.9956429 0.9912857
0.4 2 0.8 0.75 150 0.9974429 0.9948857
0.4 2 0.8 1.00 50 0.9809429 0.9618857
0.4 2 0.8 1.00 100 0.9956571 0.9913143
0.4 2 0.8 1.00 150 0.9975000 0.9950000
0.4 3 0.6 0.50 50 0.9964000 0.9928000
0.4 3 0.6 0.50 100 0.9985857 0.9971714
0.4 3 0.6 0.50 150 0.9991857 0.9983714
0.4 3 0.6 0.75 50 0.9965286 0.9930571
0.4 3 0.6 0.75 100 0.9987857 0.9975714
0.4 3 0.6 0.75 150 0.9992286 0.9984571
0.4 3 0.6 1.00 50 0.9963286 0.9926571
0.4 3 0.6 1.00 100 0.9985143 0.9970286
0.4 3 0.6 1.00 150 0.9992429 0.9984857
0.4 3 0.8 0.50 50 0.9969714 0.9939429
0.4 3 0.8 0.50 100 0.9988143 0.9976286
0.4 3 0.8 0.50 150 0.9991571 0.9983143
0.4 3 0.8 0.75 50 0.9966286 0.9932571
0.4 3 0.8 0.75 100 0.9987571 0.9975143
0.4 3 0.8 0.75 150 0.9993571 0.9987143
0.4 3 0.8 1.00 50 0.9964857 0.9929714
0.4 3 0.8 1.00 100 0.9988571 0.9977143
0.4 3 0.8 1.00 150 0.9992857 0.9985714
Tuning parameter 'gamma' was held constant at a value of 0
Tuning parameter 'min_child_weight' was held constant at
a value of 1
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were nrounds = 150, max_depth = 3, eta = 0.4, gamma = 0, colsample_bytree =
0.8, min_child_weight = 1 and subsample = 0.75.credit_xgb_pred <- predict(credit_xgb_model, test_data)
credit_xgb_cm <- confusionMatrix(credit_xgb_pred, test_data$Class)
credit_xgb_auc <- credit_xgb_cm$overall[1] *100
credit_xgb_acc <- credit_xgb_cm$overall[4] *100
credit_xgb_aucAccuracy
99.90667 credit_xgb_modeleXtreme Gradient Boosting
70000 samples
8 predictor
2 classes: 'No', 'Yes'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 63000, 63000, 63000, 63000, 63000, 63000, ...
Resampling results across tuning parameters:
eta max_depth colsample_bytree subsample nrounds Accuracy Kappa
0.3 1 0.6 0.50 50 0.9478286 0.8956571
0.3 1 0.6 0.50 100 0.9522429 0.9044857
0.3 1 0.6 0.50 150 0.9573714 0.9147429
0.3 1 0.6 0.75 50 0.9480286 0.8960571
0.3 1 0.6 0.75 100 0.9518286 0.9036571
0.3 1 0.6 0.75 150 0.9557143 0.9114286
0.3 1 0.6 1.00 50 0.9476714 0.8953429
0.3 1 0.6 1.00 100 0.9516429 0.9032857
0.3 1 0.6 1.00 150 0.9554000 0.9108000
0.3 1 0.8 0.50 50 0.9477714 0.8955429
0.3 1 0.8 0.50 100 0.9519429 0.9038857
0.3 1 0.8 0.50 150 0.9569714 0.9139429
0.3 1 0.8 0.75 50 0.9483000 0.8966000
0.3 1 0.8 0.75 100 0.9521571 0.9043143
0.3 1 0.8 0.75 150 0.9563000 0.9126000
0.3 1 0.8 1.00 50 0.9476714 0.8953429
0.3 1 0.8 1.00 100 0.9525714 0.9051429
0.3 1 0.8 1.00 150 0.9552429 0.9104857
0.3 2 0.6 0.50 50 0.9679571 0.9359143
0.3 2 0.6 0.50 100 0.9910429 0.9820857
0.3 2 0.6 0.50 150 0.9963429 0.9926857
0.3 2 0.6 0.75 50 0.9694714 0.9389429
0.3 2 0.6 0.75 100 0.9912286 0.9824571
0.3 2 0.6 0.75 150 0.9964429 0.9928857
0.3 2 0.6 1.00 50 0.9695429 0.9390857
0.3 2 0.6 1.00 100 0.9920286 0.9840571
0.3 2 0.6 1.00 150 0.9964143 0.9928286
0.3 2 0.8 0.50 50 0.9711143 0.9422286
0.3 2 0.8 0.50 100 0.9924429 0.9848857
0.3 2 0.8 0.50 150 0.9967571 0.9935143
0.3 2 0.8 0.75 50 0.9725143 0.9450286
0.3 2 0.8 0.75 100 0.9926429 0.9852857
0.3 2 0.8 0.75 150 0.9965429 0.9930857
0.3 2 0.8 1.00 50 0.9710857 0.9421714
0.3 2 0.8 1.00 100 0.9924714 0.9849429
0.3 2 0.8 1.00 150 0.9965714 0.9931429
0.3 3 0.6 0.50 50 0.9933714 0.9867429
0.3 3 0.6 0.50 100 0.9979571 0.9959143
0.3 3 0.6 0.50 150 0.9987857 0.9975714
0.3 3 0.6 0.75 50 0.9936143 0.9872286
0.3 3 0.6 0.75 100 0.9977857 0.9955714
0.3 3 0.6 0.75 150 0.9988429 0.9976857
0.3 3 0.6 1.00 50 0.9939857 0.9879714
0.3 3 0.6 1.00 100 0.9979857 0.9959714
0.3 3 0.6 1.00 150 0.9990571 0.9981143
0.3 3 0.8 0.50 50 0.9945286 0.9890571
0.3 3 0.8 0.50 100 0.9980143 0.9960286
0.3 3 0.8 0.50 150 0.9990571 0.9981143
0.3 3 0.8 0.75 50 0.9943714 0.9887429
0.3 3 0.8 0.75 100 0.9983143 0.9966286
0.3 3 0.8 0.75 150 0.9990286 0.9980571
0.3 3 0.8 1.00 50 0.9940571 0.9881143
0.3 3 0.8 1.00 100 0.9981000 0.9962000
0.3 3 0.8 1.00 150 0.9989571 0.9979143
0.4 1 0.6 0.50 50 0.9500571 0.9001143
0.4 1 0.6 0.50 100 0.9559571 0.9119143
0.4 1 0.6 0.50 150 0.9603857 0.9207714
0.4 1 0.6 0.75 50 0.9496000 0.8992000
0.4 1 0.6 0.75 100 0.9551571 0.9103143
0.4 1 0.6 0.75 150 0.9605714 0.9211429
0.4 1 0.6 1.00 50 0.9492000 0.8984000
0.4 1 0.6 1.00 100 0.9538143 0.9076286
0.4 1 0.6 1.00 150 0.9598429 0.9196857
0.4 1 0.8 0.50 50 0.9501714 0.9003429
0.4 1 0.8 0.50 100 0.9560143 0.9120286
0.4 1 0.8 0.50 150 0.9613571 0.9227143
0.4 1 0.8 0.75 50 0.9504000 0.9008000
0.4 1 0.8 0.75 100 0.9561286 0.9122571
0.4 1 0.8 0.75 150 0.9596143 0.9192286
0.4 1 0.8 1.00 50 0.9490857 0.8981714
0.4 1 0.8 1.00 100 0.9542714 0.9085429
0.4 1 0.8 1.00 150 0.9597286 0.9194571
0.4 2 0.6 0.50 50 0.9801857 0.9603714
0.4 2 0.6 0.50 100 0.9955857 0.9911714
0.4 2 0.6 0.50 150 0.9974857 0.9949714
0.4 2 0.6 0.75 50 0.9816286 0.9632571
0.4 2 0.6 0.75 100 0.9956000 0.9912000
0.4 2 0.6 0.75 150 0.9974286 0.9948571
0.4 2 0.6 1.00 50 0.9815143 0.9630286
0.4 2 0.6 1.00 100 0.9956000 0.9912000
0.4 2 0.6 1.00 150 0.9975714 0.9951429
0.4 2 0.8 0.50 50 0.9822714 0.9645429
0.4 2 0.8 0.50 100 0.9955857 0.9911714
0.4 2 0.8 0.50 150 0.9973857 0.9947714
0.4 2 0.8 0.75 50 0.9811286 0.9622571
0.4 2 0.8 0.75 100 0.9956429 0.9912857
0.4 2 0.8 0.75 150 0.9974429 0.9948857
0.4 2 0.8 1.00 50 0.9809429 0.9618857
0.4 2 0.8 1.00 100 0.9956571 0.9913143
0.4 2 0.8 1.00 150 0.9975000 0.9950000
0.4 3 0.6 0.50 50 0.9964000 0.9928000
0.4 3 0.6 0.50 100 0.9985857 0.9971714
0.4 3 0.6 0.50 150 0.9991857 0.9983714
0.4 3 0.6 0.75 50 0.9965286 0.9930571
0.4 3 0.6 0.75 100 0.9987857 0.9975714
0.4 3 0.6 0.75 150 0.9992286 0.9984571
0.4 3 0.6 1.00 50 0.9963286 0.9926571
0.4 3 0.6 1.00 100 0.9985143 0.9970286
0.4 3 0.6 1.00 150 0.9992429 0.9984857
0.4 3 0.8 0.50 50 0.9969714 0.9939429
0.4 3 0.8 0.50 100 0.9988143 0.9976286
0.4 3 0.8 0.50 150 0.9991571 0.9983143
0.4 3 0.8 0.75 50 0.9966286 0.9932571
0.4 3 0.8 0.75 100 0.9987571 0.9975143
0.4 3 0.8 0.75 150 0.9993571 0.9987143
0.4 3 0.8 1.00 50 0.9964857 0.9929714
0.4 3 0.8 1.00 100 0.9988571 0.9977143
0.4 3 0.8 1.00 150 0.9992857 0.9985714
Tuning parameter 'gamma' was held constant at a value of 0
Tuning parameter 'min_child_weight' was held constant at
a value of 1
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were nrounds = 150, max_depth = 3, eta = 0.4, gamma = 0, colsample_bytree =
0.8, min_child_weight = 1 and subsample = 0.75.credit_xgb_cmConfusion Matrix and Statistics
Reference
Prediction No Yes
No 14972 0
Yes 28 15000
Accuracy : 0.9991
95% CI : (0.9987, 0.9994)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.9981
Mcnemar's Test P-Value : 3.352e-07
Sensitivity : 0.9981
Specificity : 1.0000
Pos Pred Value : 1.0000
Neg Pred Value : 0.9981
Prevalence : 0.5000
Detection Rate : 0.4991
Detection Prevalence : 0.4991
Balanced Accuracy : 0.9991
'Positive' Class : No
train_data$Class=as.numeric(train_data$Class)-1
test_data$Class=as.numeric(test_data$Class)-1head(test_data)head(train_data)normalize <- function(x) {
return ((x - min(x)) / (max(x) - min(x)))
}
scale.cols <- c("V3","V4","V7","V10","V12","V14","V16","V17")
train_data[scale.cols] <- lapply(train_data[scale.cols], normalize)
test_data[scale.cols] <- lapply(test_data[scale.cols], normalize)
train_datatest_data#Neural Network Model
set.seed(1)
credit_train_idex1<- createDataPartition(train_data$Class, p = 0.9, list = FALSE)
credit_card_train = train_data[credit_train_idex1,]
credit_card_valid = train_data[-credit_train_idex1,]
credit_card_train_x <- credit_card_train[, -9]
credit_card_train_y <- credit_card_train[, 9]
credit_card_valid_x <- credit_card_valid[,-9]
credit_card_valid_y <- credit_card_valid[, 9]
credit_card_test_x <-test_data[, -9]
credit_card_test_y <- test_data[, 9]library(keras)
Attaching package: ‘keras’
The following object is masked _by_ ‘.GlobalEnv’:
normalizeset.seed(1)
model <- keras_model_sequential() %>%
layer_dense(units = 64, activation = "relu", input_shape = as.matrix(dim(credit_card_train_x)[2])) %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 1, activation = "sigmoid")Metal device set to: Apple M2
systemMemory: 8.00 GB
maxCacheSize: 2.67 GBmodel %>% compile(
loss = "binary_crossentropy",
optimizer = optimizer_adam(learning_rate = 0.001),
metrics = "accuracy"
)WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.history <- model %>% fit(
as.matrix(credit_card_train_x),
credit_card_train_y,
batch_size = 50,
epochs = 20,
verbose = 2,
validation_data = list(as.matrix(credit_card_valid_x), credit_card_valid_y)
)2023-05-09 19:17:03.803903: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 HzEpoch 1/20
1260/1260 - 6s - loss: 0.2170 - accuracy: 0.9220 - val_loss: 0.1770 - val_accuracy: 0.9316 - 6s/epoch - 5ms/step
Epoch 2/20
1260/1260 - 5s - loss: 0.1709 - accuracy: 0.9348 - val_loss: 0.1650 - val_accuracy: 0.9381 - 5s/epoch - 4ms/step
Epoch 3/20
1260/1260 - 5s - loss: 0.1657 - accuracy: 0.9376 - val_loss: 0.1649 - val_accuracy: 0.9393 - 5s/epoch - 4ms/step
Epoch 4/20
1260/1260 - 5s - loss: 0.1656 - accuracy: 0.9377 - val_loss: 0.1637 - val_accuracy: 0.9393 - 5s/epoch - 4ms/step
Epoch 5/20
1260/1260 - 5s - loss: 0.1646 - accuracy: 0.9380 - val_loss: 0.1753 - val_accuracy: 0.9359 - 5s/epoch - 4ms/step
Epoch 6/20
1260/1260 - 5s - loss: 0.1639 - accuracy: 0.9382 - val_loss: 0.1695 - val_accuracy: 0.9356 - 5s/epoch - 4ms/step
Epoch 7/20
1260/1260 - 5s - loss: 0.1622 - accuracy: 0.9384 - val_loss: 0.1618 - val_accuracy: 0.9413 - 5s/epoch - 4ms/step
Epoch 8/20
1260/1260 - 5s - loss: 0.1628 - accuracy: 0.9380 - val_loss: 0.1629 - val_accuracy: 0.9403 - 5s/epoch - 4ms/step
Epoch 9/20
1260/1260 - 5s - loss: 0.1618 - accuracy: 0.9383 - val_loss: 0.1599 - val_accuracy: 0.9407 - 5s/epoch - 4ms/step
Epoch 10/20
1260/1260 - 5s - loss: 0.1602 - accuracy: 0.9385 - val_loss: 0.1686 - val_accuracy: 0.9343 - 5s/epoch - 4ms/step
Epoch 11/20
1260/1260 - 5s - loss: 0.1597 - accuracy: 0.9390 - val_loss: 0.1583 - val_accuracy: 0.9419 - 5s/epoch - 4ms/step
Epoch 12/20
1260/1260 - 5s - loss: 0.1591 - accuracy: 0.9391 - val_loss: 0.1608 - val_accuracy: 0.9409 - 5s/epoch - 4ms/step
Epoch 13/20
1260/1260 - 5s - loss: 0.1582 - accuracy: 0.9398 - val_loss: 0.1560 - val_accuracy: 0.9420 - 5s/epoch - 4ms/step
Epoch 14/20
1260/1260 - 5s - loss: 0.1571 - accuracy: 0.9397 - val_loss: 0.1554 - val_accuracy: 0.9427 - 5s/epoch - 4ms/step
Epoch 15/20
1260/1260 - 5s - loss: 0.1575 - accuracy: 0.9395 - val_loss: 0.1575 - val_accuracy: 0.9401 - 5s/epoch - 4ms/step
Epoch 16/20
1260/1260 - 5s - loss: 0.1556 - accuracy: 0.9404 - val_loss: 0.1569 - val_accuracy: 0.9406 - 5s/epoch - 4ms/step
Epoch 17/20
1260/1260 - 5s - loss: 0.1555 - accuracy: 0.9403 - val_loss: 0.1561 - val_accuracy: 0.9420 - 5s/epoch - 4ms/step
Epoch 18/20
1260/1260 - 5s - loss: 0.1547 - accuracy: 0.9400 - val_loss: 0.1562 - val_accuracy: 0.9400 - 5s/epoch - 4ms/step
Epoch 19/20
1260/1260 - 5s - loss: 0.1540 - accuracy: 0.9411 - val_loss: 0.1515 - val_accuracy: 0.9420 - 5s/epoch - 4ms/step
Epoch 20/20
1260/1260 - 5s - loss: 0.1527 - accuracy: 0.9412 - val_loss: 0.1514 - val_accuracy: 0.9406 - 5s/epoch - 4ms/steppredicted_probs <- model %>% predict(as.matrix(credit_card_test_x))
1/938 [..............................] - ETA: 46s
46/938 [>.............................] - ETA: 0s
88/938 [=>............................] - ETA: 0s
133/938 [===>..........................] - ETA: 0s
179/938 [====>.........................] - ETA: 0s
227/938 [======>.......................] - ETA: 0s
275/938 [=======>......................] - ETA: 0s
322/938 [=========>....................] - ETA: 0s
371/938 [==========>...................] - ETA: 0s
419/938 [============>.................] - ETA: 0s
466/938 [=============>................] - ETA: 0s
513/938 [===============>..............] - ETA: 0s
559/938 [================>.............] - ETA: 0s
608/938 [==================>...........] - ETA: 0s
655/938 [===================>..........] - ETA: 0s
703/938 [=====================>........] - ETA: 0s
750/938 [======================>.......] - ETA: 0s
797/938 [========================>.....] - ETA: 0s
844/938 [=========================>....] - ETA: 0s
893/938 [===========================>..] - ETA: 0s
938/938 [==============================] - 1s 1ms/step
938/938 [==============================] - 1s 1ms/stepstr(predicted_probs) num [1:30000, 1] 0.00629 0.99777 0.19117 1 0.01484 ...
ann_predictions= prediction(predicted_probs,credit_card_test_y)
ann_predictionsA prediction instance
with 30000 data pointsperformance(ann_predictions, measure = "auc")@y.values[[1]]
[1] 0.9790121plot(history)
predicted.labels = factor(ifelse(predicted_probs>0.5, "1", "0"))
confusionMatrix(predicted.labels, as.factor(credit_card_test_y),mode="everything", positive="1")Confusion Matrix and Statistics
Reference
Prediction 0 1
0 14922 2095
1 78 12905
Accuracy : 0.9276
95% CI : (0.9246, 0.9305)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.8551
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 0.8603
Specificity : 0.9948
Pos Pred Value : 0.9940
Neg Pred Value : 0.8769
Precision : 0.9940
Recall : 0.8603
F1 : 0.9223
Prevalence : 0.5000
Detection Rate : 0.4302
Detection Prevalence : 0.4328
Balanced Accuracy : 0.9276
'Positive' Class : 1
The Accuracy of Neural Network Model is 90.7
modelModel: "sequential"
______________________________________________________________________________________________________________________
Layer (type) Output Shape Param #
======================================================================================================================
dense_2 (Dense) (None, 64) 576
dense_1 (Dense) (None, 64) 4160
dense (Dense) (None, 1) 65
======================================================================================================================
Total params: 4,801
Trainable params: 4,801
Non-trainable params: 0
______________________________________________________________________________________________________________________#Neural Network Model With Classweights.
library(keras)
model_weight <- keras_model_sequential() %>%
layer_dense(units = 64, activation = "relu", input_shape = as.matrix(dim(credit_card_train_x)[2])) %>%
layer_dense(units = 64, activation = "relu") %>%
layer_dense(units = 1, activation = "sigmoid")
model_weight %>% compile(
loss = "binary_crossentropy",
optimizer = optimizer_adam(learning_rate = 0.001),
metrics = "accuracy"
)WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.history_model <- model_weight %>% fit(
as.matrix(credit_card_train_x),
credit_card_train_y,
batch_size = 50,
epochs = 20,
verbose = 2,
validation_data = list(as.matrix(credit_card_valid_x), credit_card_valid_y),class_weight=list("0", "1")
)Epoch 1/20
1260/1260 - 6s - loss: 0.2212 - accuracy: 0.9148 - val_loss: 0.1861 - val_accuracy: 0.9326 - 6s/epoch - 5ms/step
Epoch 2/20
1260/1260 - 5s - loss: 0.1701 - accuracy: 0.9353 - val_loss: 0.1773 - val_accuracy: 0.9363 - 5s/epoch - 4ms/step
Epoch 3/20
1260/1260 - 5s - loss: 0.1649 - accuracy: 0.9367 - val_loss: 0.1621 - val_accuracy: 0.9403 - 5s/epoch - 4ms/step
Epoch 4/20
1260/1260 - 5s - loss: 0.1629 - accuracy: 0.9380 - val_loss: 0.1625 - val_accuracy: 0.9411 - 5s/epoch - 4ms/step
Epoch 5/20
1260/1260 - 5s - loss: 0.1628 - accuracy: 0.9378 - val_loss: 0.1603 - val_accuracy: 0.9413 - 5s/epoch - 4ms/step
Epoch 6/20
1260/1260 - 5s - loss: 0.1614 - accuracy: 0.9382 - val_loss: 0.1591 - val_accuracy: 0.9409 - 5s/epoch - 4ms/step
Epoch 7/20
1260/1260 - 5s - loss: 0.1621 - accuracy: 0.9383 - val_loss: 0.1668 - val_accuracy: 0.9374 - 5s/epoch - 4ms/step
Epoch 8/20
1260/1260 - 5s - loss: 0.1606 - accuracy: 0.9391 - val_loss: 0.1600 - val_accuracy: 0.9401 - 5s/epoch - 4ms/step
Epoch 9/20
1260/1260 - 5s - loss: 0.1596 - accuracy: 0.9394 - val_loss: 0.1615 - val_accuracy: 0.9391 - 5s/epoch - 4ms/step
Epoch 10/20
1260/1260 - 5s - loss: 0.1596 - accuracy: 0.9392 - val_loss: 0.1598 - val_accuracy: 0.9403 - 5s/epoch - 4ms/step
Epoch 11/20
1260/1260 - 5s - loss: 0.1585 - accuracy: 0.9392 - val_loss: 0.1700 - val_accuracy: 0.9406 - 5s/epoch - 4ms/step
Epoch 12/20
1260/1260 - 5s - loss: 0.1570 - accuracy: 0.9394 - val_loss: 0.1573 - val_accuracy: 0.9381 - 5s/epoch - 4ms/step
Epoch 13/20
1260/1260 - 5s - loss: 0.1561 - accuracy: 0.9402 - val_loss: 0.1558 - val_accuracy: 0.9414 - 5s/epoch - 4ms/step
Epoch 14/20
1260/1260 - 5s - loss: 0.1568 - accuracy: 0.9399 - val_loss: 0.1552 - val_accuracy: 0.9414 - 5s/epoch - 4ms/step
Epoch 15/20
1260/1260 - 5s - loss: 0.1546 - accuracy: 0.9401 - val_loss: 0.1529 - val_accuracy: 0.9391 - 5s/epoch - 4ms/step
Epoch 16/20
1260/1260 - 5s - loss: 0.1534 - accuracy: 0.9406 - val_loss: 0.1512 - val_accuracy: 0.9416 - 5s/epoch - 4ms/step
Epoch 17/20
1260/1260 - 5s - loss: 0.1526 - accuracy: 0.9407 - val_loss: 0.1513 - val_accuracy: 0.9427 - 5s/epoch - 4ms/step
Epoch 18/20
1260/1260 - 5s - loss: 0.1508 - accuracy: 0.9415 - val_loss: 0.1502 - val_accuracy: 0.9423 - 5s/epoch - 4ms/step
Epoch 19/20
1260/1260 - 5s - loss: 0.1496 - accuracy: 0.9421 - val_loss: 0.1484 - val_accuracy: 0.9436 - 5s/epoch - 4ms/step
Epoch 20/20
1260/1260 - 5s - loss: 0.1492 - accuracy: 0.9419 - val_loss: 0.1554 - val_accuracy: 0.9389 - 5s/epoch - 4ms/stepplot(history_model)
predicted_probs_weight <- model_weight %>% predict(as.matrix(credit_card_test_x))
1/938 [..............................] - ETA: 32s
46/938 [>.............................] - ETA: 0s
93/938 [=>............................] - ETA: 0s
140/938 [===>..........................] - ETA: 0s
184/938 [====>.........................] - ETA: 0s
231/938 [======>.......................] - ETA: 0s
279/938 [=======>......................] - ETA: 0s
325/938 [=========>....................] - ETA: 0s
367/938 [==========>...................] - ETA: 0s
414/938 [============>.................] - ETA: 0s
463/938 [=============>................] - ETA: 0s
511/938 [===============>..............] - ETA: 0s
558/938 [================>.............] - ETA: 0s
606/938 [==================>...........] - ETA: 0s
654/938 [===================>..........] - ETA: 0s
694/938 [=====================>........] - ETA: 0s
739/938 [======================>.......] - ETA: 0s
785/938 [========================>.....] - ETA: 0s
832/938 [=========================>....] - ETA: 0s
878/938 [===========================>..] - ETA: 0s
925/938 [============================>.] - ETA: 0s
938/938 [==============================] - 1s 1ms/step
938/938 [==============================] - 1s 1ms/stepstr(predicted_probs_weight) num [1:30000, 1] 0.00487 0.99236 0.3171 1 0.02441 ...ann_predictions_weight= prediction(predicted_probs,credit_card_test_y)
ann_predictions_weightA prediction instance
with 30000 data pointsperformance(ann_predictions_weight, measure = "auc")@y.values[[1]]
[1] 0.9790121range(credit_card_test_x)[1] 0 1predicted.labels_weight = factor(ifelse(predicted_probs_weight>0.5, "1", "0"))
confusionMatrix(predicted.labels_weight, as.factor(credit_card_test_y),mode="everything", positive="1")Confusion Matrix and Statistics
Reference
Prediction 0 1
0 14901 2106
1 99 12894
Accuracy : 0.9265
95% CI : (0.9235, 0.9294)
No Information Rate : 0.5
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.853
Mcnemar's Test P-Value : < 2.2e-16
Sensitivity : 0.8596
Specificity : 0.9934
Pos Pred Value : 0.9924
Neg Pred Value : 0.8762
Precision : 0.9924
Recall : 0.8596
F1 : 0.9212
Prevalence : 0.5000
Detection Rate : 0.4298
Detection Prevalence : 0.4331
Balanced Accuracy : 0.9265
'Positive' Class : 1
#Summary of Accuracy
Random Forest: 99.99%
DecisionTreeModel using C50: 99.97%
XGBoost: 99.93797%
KNN Smote: 99.38%
KNN: 99.38%
Gradient Boosted Tree model: 97.17%
SVM Radical: 97.04%
NeuralNetwork with class weight: 92.64%
DecisionTreeModel using rpart: 93.50195%
SVM Smote: 94.21%
SVM: 94.21%
Neural Network Model: 90.7%
LS0tCnRpdGxlOiAiQ3JlZGl0IENhcmQgRnJhdWQgRGV0ZWN0aW9uIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgojIFBST0pFQ1QgVE9QSUMgLSAgQ1JFRElUIENBUkQgRlJBVUQgREVURUNUSU9OCgojMS4gRGF0YSBEZXNjcmlwdGlvbjoKVGltZS0gTnVtYmVyIG9mIHNlY29uZHMgZWxhcHNlZCBiZXR3ZWVuIHRoaXMgdHJhbnNhY3Rpb24gYW5kIHRoZSBmaXJzdCB0cmFuc2FjdGlvbiBpbiB0aGUgZGF0YXNldApWMS1WMjggLSBSZXN1bHQgb2YgYSBQQ0EgRGltZW5zaW9uYWxpdHkgcmVkdWN0aW9uIHRvIHByb3RlY3QgdXNlciBpZGVudGl0aWVzIGFuZCBzZW5zaXRpdmUgZmVhdHVyZXModjEtdjI4KQpBbW91bnQgLSBUcmFuc2FjdGlvbiBhbW91bnQKQ2xhc3MgLSAxIGZvciBmcmF1ZHVsZW50IHRyYW5zYWN0aW9ucywgMCBvdGhlcndpc2UKCiNHb2FsCgpUbyBpZGVudGlmeSBmcmF1ZHVsZW50IHRyYW5zYWN0aW9ucyAgd2hpY2ggY2FuIHJlc3VsdCBpbiBzaWduaWZpY2FudCBmaW5hbmNpYWwgbG9zc2VzIGZvciBpbmRpdmlkdWFscyBhbmQgYnVzaW5lc3Nlcy4gVGhlIGdvYWwgaXMgdG8gZGV2ZWxvcCBhIG1hY2hpbmUgbGVhcm5pbmcgbW9kZWwgdGhhdCBjYW4gYWNjdXJhdGVseSBjbGFzc2lmeSB0cmFuc2FjdGlvbnMgYXMgZWl0aGVyIGZyYXVkdWxlbnQgb3IgbGVnaXRpbWF0ZSwgYmFzZWQgb24gdmFyaW91cyBmZWF0dXJlcyBzdWNoIGFzIHRyYW5zYWN0aW9uIGFtb3VudCwgbG9jYXRpb24sIGFuZCB0aW1lLgoKI0RhdGFzZXQgUmVmZXJlbmNlOgoKVGhlIENyZWRpdCBDYXJkIEZyYXVkIERldGVjdGlvbiBkYXRhc2V0IG9uIEthZ2dsZSBhdCB0aGUgZm9sbG93aW5nIGxpbmsgaHR0cHM6Ly93d3cua2FnZ2xlLmNvbS9tbGctdWxiL2NyZWRpdGNhcmRmcmF1ZCAgTGlua3MgdG8gYW4gZXh0ZXJuYWwgc2l0ZS4KVGhlIGRhdGFzZXQgaGFzIDMxIHZhcmlhYmxlcywgd2l0aCB0aGUgdGFyZ2V0IHZhcmlhYmxlIGluZGljYXRpbmcgd2hldGhlciBhIHRyYW5zYWN0aW9uIGlzIGZyYXVkdWxlbnQgb3Igbm90LgoKI01vZGVsOgoKSSB3aWxsIGJlIHVzaW5nIGEgY2xhc3NpZmljYXRpb24gbW9kZWwgdG8gc29sdmUgdGhlIHByb2JsZW0gb2YgY3JlZGl0IGNhcmQgZnJhdWQgZGV0ZWN0aW9uLiBUaGUgb3V0Y29tZSB2YXJpYWJsZSB3aWxsIGJlIGEgYmluYXJ5IHZhcmlhYmxlIGluZGljYXRpbmcgd2hldGhlciBhIGNyZWRpdCBjYXJkIHRyYW5zYWN0aW9uIGlzIGZyYXVkdWxlbnQgb3Igbm90LgoKYGBge3J9CmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KGNhcmV0KQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkobGF0dGljZSkKbGlicmFyeShncmlkRXh0cmEpCmxpYnJhcnkoY29ycnBsb3QpCmxpYnJhcnkoUk9DUikKYGBgCgoKIzEgTG9hZGluZyB0aGUgZGF0YXNldDoKYGBge3J9CmNyZWRpdF9jYXJkIDwtIHJlYWQuY3N2KCIvVXNlcnMvdmlub3RoL0Rvd25sb2Fkcy9jcmVkaXRjYXJkLmNzdiIpCmNyZWRpdF9jYXJkJENsYXNzID0gZmFjdG9yKGNyZWRpdF9jYXJkJENsYXNzLGxldmVscyA9IGMoMCwxKSkKYGBgCgojIEdldHRpbmcgdGhlIHN1bW1hcnkgb2Ygc3RydWN0dXJlIG9mIGRhdGFzZXQKYGBge3J9CmhlYWQoY3JlZGl0X2NhcmQpCmRpbShjcmVkaXRfY2FyZCkKc3RyKGNyZWRpdF9jYXJkKQpzdW1tYXJ5KGNyZWRpdF9jYXJkKQpgYGAKIyBVc2luZyBkdXBsaWNhdGVkIGZ1bmN0aW9uIHRvIGR1cGxpY2F0ZWQgZWxlbWVudHMuCmBgYHtyfQpkdXBsaWNhdGVkKGNyZWRpdF9jYXJkKQpgYGAKCgojTnVtYmVyIG9mIG1pc3NpbmcgdmFsdWVzIGluIHJlYWx0b3IgZGF0YXNldC4KYGBge3J9CnN1bShpcy5uYShjcmVkaXRfY2FyZCkpCmBgYAoKVGhlcmUgYXJlIG5vIG1pc3NpbmcgdmFsdWVzIGluIHRoZSBkYXRhc2V0LgoKCiNDbGFzcyBEaXN0cmlidXRpb246IAoKYGBge3J9CnRhYmxlKGNyZWRpdF9jYXJkJENsYXNzKQpiYXJwbG90KHRhYmxlKGNyZWRpdF9jYXJkJENsYXNzKSkKYGBgCgojVGltZSBhbmQgQW1vdW50IEFuYWx5c2lzOiAKYGBge3J9Cmhpc3QoY3JlZGl0X2NhcmQkVGltZSkKaGlzdChjcmVkaXRfY2FyZCRBbW91bnQpCmBgYApgYGB7cn0KYm94cGxvdChBbW91bnQgfiBDbGFzcywgZGF0YSA9IGNyZWRpdF9jYXJkLCB4bGFiID0gIkNsYXNzIiwgeWxhYiA9ICJBbW91bnQiKQpwbG90KGNyZWRpdF9jYXJkJFYxLCBjcmVkaXRfY2FyZCRWMiwgY29sID0gaWZlbHNlKGNyZWRpdF9jYXJkJENsYXNzID09IDEsICJyZWQiLCAiYmx1ZSIpLHhsYWIgPSAiVjEiLCB5bGFiID0gIlYyIikKYGBgCgojRmluZGluZyB0aGUgQ29ycmVsYXRpb24gb2YgZXZlcnkgZmVhdHVyZSB2cyBDbGFzcyBGZWF0dXJlLgpgYGB7cn0KY29ycmVsYXRpb24gPC0gY29yKGNyZWRpdF9jYXJkWywtMzFdLCBhcy5udW1lcmljKGNyZWRpdF9jYXJkWywzMV0pKQpjb3JyZWxhdGlvbgpgYGAKVGhlIGNvbHVtbnMgbGFiZWxlZCBhcyB2MSwgdjMsIHYxMCwgdjEyLCB2MTQsIHYxNiwgdjE3LCBhbmQgdjE4IGV4aGliaXQgYSBuZWdhdGl2ZSBjb3JyZWxhdGlvbiwgaW5kaWNhdGluZyBhIHN0cm9uZyBpbnZlcnNlIHJlbGF0aW9uc2hpcC4gQ29udmVyc2VseSwgdGhlIGNvbHVtbnMgdjIsIHY0LCBhbmQgdjExIGRlbW9uc3RyYXRlIGEgcG9zaXRpdmUgY29ycmVsYXRpb24sIGluZGljYXRpbmcgYSBkaXJlY3QgcmVsYXRpb25zaGlwLiBUaGUgcmVtYWluaW5nIGNvbHVtbnMgZG8gbm90IGRpc3BsYXkgYSBzaWduaWZpY2FudCBjb3JyZWxhdGlvbiB3aXRoIHRoZSBjbGFzcyB2YXJpYWJsZSwgc3VnZ2VzdGluZyB0aGF0IHRoZXkgYXJlIG5vdCBzdHJvbmdseSBhc3NvY2lhdGVkIHdpdGggaXQuCgpgYGB7cn0KY29ycnBsb3QoY29ycmVsYXRpb24sIG1ldGhvZCA9ICJwaWUiLCBjbC5wb3M9J24nKQpgYGAKCk9uIHRoZSBBYm92ZSBvYnNlcnZhdGlvbiB3ZSBjYW4gY29uY2x1ZGUgdGhhdDoKClZhcmlhYmxlcyB3aXRoIFN0cm9uZyBOZWdhdGl2ZSBDb3JyZWxhdGlvbnM6ClYxMCAoLTAuMjE2ODgyOTQzNikKVjEyICgtMC4yNjA1OTI5MjQ5KQpWMTQgKC0wLjMwMjU0MzY5NTgpClYxNyAoLTAuMzI2NDgxMDY3MikKVmFyaWFibGVzIHdpdGggTW9kZXJhdGUgTmVnYXRpdmUgQ29ycmVsYXRpb25zOgpWMyAoLTAuMTkyOTYwODI3MSkKVjcgKC0wLjE4NzI1NjU5MTUpClYxNiAoLTAuMTk2NTM4OTQwMykKVmFyaWFibGVzIHdpdGggTW9kZXJhdGUgUG9zaXRpdmUgQ29ycmVsYXRpb25zOgpWNCAoMC4xMzM0NDc0ODYyKQoKI3JlbW92aW5nIGxlc3MgY29ycmVsYXRlZCB2YXJpYWJsZXMuCmBgYHtyfQpjcmVkaXRfY2FyZCA8LSBzZWxlY3QoY3JlZGl0X2NhcmQsIFYzLCBWNCwgVjcsIFYxMCwgVjEyLCBWMTQsIFYxNiwgVjE3LCBDbGFzcykKY3JlZGl0X2NhcmQKYGBgCgpgYGB7cn0Kc3VtbWFyeShjcmVkaXRfY2FyZCkKYGBgCgojQm94IFBsb3QsIERlbnNpdHkgcGxvdCBmb3IgYWxsIEZlYXR1cmUgdnMgQ2xhc3MKCmBgYHtyfQpib3hwbG90X3Bsb3RzIDwtIGxpc3QoKQpkZW5zaXR5X3Bsb3RzIDwtIGxpc3QoKQoKZm9yICh2YXIgaW4gYygiVjMiLCAiVjQiLCAiVjciLCAiVjEwIiwgIlYxMiIsICJWMTQiLCAiVjE2IiwgIlYxNyIpKSB7CgogIGJveHBsb3RfcGxvdCA8LSBnZ3Bsb3QoY3JlZGl0X2NhcmQsIGFlcyh4ID0gZmFjdG9yKENsYXNzKSwgeSA9IC5kYXRhW1t2YXJdXSwgZmlsbCA9IGZhY3RvcihDbGFzcykpKSArCiAgICBnZW9tX2JveHBsb3QoKSArCiAgICBsYWJzKHRpdGxlID0gcGFzdGUoIkJveHBsb3Qgb2YiLCB2YXIsICJieSBDbGFzcyIpKSArCiAgICB0aGVtZV9taW5pbWFsKCkKCiAgICBib3hwbG90X3Bsb3RzIDwtIGFwcGVuZChib3hwbG90X3Bsb3RzLCBsaXN0KGJveHBsb3RfcGxvdCkpCgogIGRlbnNpdHlfcGxvdCA8LSBnZ3Bsb3QoY3JlZGl0X2NhcmQsIGFlcyh4ID0gLmRhdGFbW3Zhcl1dLCBmaWxsID0gZmFjdG9yKENsYXNzKSkpICsKICAgIGdlb21fZGVuc2l0eShhbHBoYSA9IDAuNykgKwogICAgbGFicyh0aXRsZSA9IHBhc3RlKCJEZW5zaXR5IFBsb3Qgb2YiLCB2YXIsICJieSBDbGFzcyIpKSArCiAgICB0aGVtZV9taW5pbWFsKCkKICAKICAgZGVuc2l0eV9wbG90cyA8LSBhcHBlbmQoZGVuc2l0eV9wbG90cywgbGlzdChkZW5zaXR5X3Bsb3QpKQp9Cgpmb3IgKHBsb3QgaW4gYm94cGxvdF9wbG90cykgewogIHByaW50KHBsb3QpCn0KCmZvciAocGxvdCBpbiBkZW5zaXR5X3Bsb3RzKSB7CiAgcHJpbnQocGxvdCkKfQpgYGAKCgpgYGB7cn0KI3QtdGVzdApmb3IgKHZhciBpbiBjKCJWMyIsICJWNCIsICJWNyIsICJWMTAiLCAiVjEyIiwgIlYxNCIsICJWMTYiLCAiVjE3IikpIHsKICBjbGFzc18wIDwtIGNyZWRpdF9jYXJkW1t2YXJdXVtjcmVkaXRfY2FyZCRDbGFzcyA9PSAwXQogIGNsYXNzXzEgPC0gY3JlZGl0X2NhcmRbW3Zhcl1dW2NyZWRpdF9jYXJkJENsYXNzID09IDFdCiAgdF90ZXN0IDwtIHQudGVzdChjbGFzc18wLCBjbGFzc18xKQogIGNhdChwYXN0ZSgiVmFyaWFibGU6IiwgdmFyLCAiXG4iKSkKICBjYXQocGFzdGUoInQtdGVzdCBwLXZhbHVlOiIsIHRfdGVzdCRwLnZhbHVlLCAiXG5cbiIpKQp9CmBgYAoKI1VuZGVyIFNhbXBsaW5nIGFuZCBPdmVyIFNhbXBsaW5nIG9mIGRhdGEKCmBgYHtyfQpjcmVkaXRfY2FyZF8wIDwtIGNyZWRpdF9jYXJkW2NyZWRpdF9jYXJkJENsYXNzID09IDAsIF0KY3JlZGl0X2NhcmRfMSA8LSBjcmVkaXRfY2FyZFtjcmVkaXRfY2FyZCRDbGFzcyA9PSAxLCBdCgojVW5kZXJzYW1wbGluZwpjcmVkaXRfY2FyZF8wX2Rvd24gPC0gY3JlZGl0X2NhcmRfMFtzYW1wbGUobnJvdyhjcmVkaXRfY2FyZF8wKSwgNTAwMDApLCBdCgojIE92ZXJzYW1wbGluZwpjcmVkaXRfY2FyZF8xX292ZXIgPC0gY3JlZGl0X2NhcmRfMVtzYW1wbGUobnJvdyhjcmVkaXRfY2FyZF8xKSwgNTAwMDAsIHJlcGxhY2UgPSBUUlVFKSwgXQoKI0NvbWJpbmluZwpjcmVkaXRfY2FyZCA8LSByYmluZChjcmVkaXRfY2FyZF8wX2Rvd24sIGNyZWRpdF9jYXJkXzFfb3ZlcikKCmNyZWRpdF9jYXJkCmBgYAoKYGBge3J9CmRpbShjcmVkaXRfY2FyZCkKc3VtbWFyeShjcmVkaXRfY2FyZCkKYGBgCgojUmFkb21pemluZyB0aGUgZGF0YSBiZWZvcmUgc3BsaXRpbmcKYGBge3J9CmNyZWRpdF9jYXJkIDwtIGNyZWRpdF9jYXJkW3NhbXBsZShucm93KGNyZWRpdF9jYXJkKSksIF0KY3JlZGl0X2NhcmQKYGBgCgojU3BsaXRpbmcgdGhlIGRhdGEgaW50byB0cmFpbmluZyBhbmQgdGVzdGluZyBzZXRzCmBgYHtyfQpsZXZlbHMoY3JlZGl0X2NhcmQkQ2xhc3MpPWxpc3QoTm8gPSIwIiwgWWVzPSIxIikKc2V0LnNlZWQoMTIzKQp0cmFpbl9pbmRpY2VzIDwtIGNyZWF0ZURhdGFQYXJ0aXRpb24oY3JlZGl0X2NhcmQkQ2xhc3MsIHAgPSAwLjcsIGxpc3QgPSBGQUxTRSkKdHJhaW5fZGF0YSA8LSBjcmVkaXRfY2FyZFt0cmFpbl9pbmRpY2VzLCBdCnRlc3RfZGF0YSA8LSBjcmVkaXRfY2FyZFstdHJhaW5faW5kaWNlcywgXQoKYGBgCgpgYGB7cn0KIyBDaGVjayB0aGUgZGltZW5zaW9ucyBvZiB0aGUgdHJhaW4gYW5kIHRlc3Qgc2V0cwpkaW0odHJhaW5fZGF0YSkKZGltKHRlc3RfZGF0YSkKc3RyKHRyYWluX2RhdGEpCnN0cih0ZXN0X2RhdGEpCmBgYAojIERlY2lzaW9uIHRyZWUgdXNpbmcgYzUwLgpgYGB7cn0KbGlicmFyeShDNTApCgptb2RlbF9ib29zdGVkIDwtIEM1LjAoQ2xhc3MgfiAuLCBkYXRhID0gdHJhaW5fZGF0YSwgdHJpYWxzID0gMzApCgpwcmVkX2Jvb3N0ZWQgPC0gcHJlZGljdChtb2RlbF9ib29zdGVkLCBuZXdkYXRhID0gdGVzdF9kYXRhLCB0eXBlID0gImNsYXNzIikKCmNvbmZ1c2lvbl9tYXRyaXhfYm9vc3RlZCA8LSB0YWJsZSh0ZXN0X2RhdGEkQ2xhc3MsIHByZWRfYm9vc3RlZCkKCmNvbmZ1c2lvbl9tYXRyaXhfYm9vc3RlZAoKYWNjdXJhY3lfYm9vc3RlZCA8LSBzdW0oZGlhZyhjb25mdXNpb25fbWF0cml4X2Jvb3N0ZWQpKSAvIHN1bShjb25mdXNpb25fbWF0cml4X2Jvb3N0ZWQpCgp0b3RhbF9lcnJvcl9ib29zdGVkIDwtIDEgLSBhY2N1cmFjeV9ib29zdGVkCgpwcmVjaXNpb25fYm9vc3RlZCA8LSBkaWFnKGNvbmZ1c2lvbl9tYXRyaXhfYm9vc3RlZCkgLyBjb2xTdW1zKGNvbmZ1c2lvbl9tYXRyaXhfYm9vc3RlZCkKcmVjYWxsX2Jvb3N0ZWQgPC0gZGlhZyhjb25mdXNpb25fbWF0cml4X2Jvb3N0ZWQpIC8gcm93U3Vtcyhjb25mdXNpb25fbWF0cml4X2Jvb3N0ZWQpCgpwcmVjaXNpb25fcmVjYWxsX2Jvb3N0ZWQgPC0gZGF0YS5mcmFtZShQcmVjaXNpb24gPSBwcmVjaXNpb25fYm9vc3RlZCwgUmVjYWxsID0gcmVjYWxsX2Jvb3N0ZWQpCgpwcmVjaXNpb25fcmVjYWxsX3RvdGFsX2Jvb3N0ZWQgPC0gcmJpbmQocHJlY2lzaW9uX3JlY2FsbF9ib29zdGVkLCBjKG1lYW4ocHJlY2lzaW9uX2Jvb3N0ZWQpLCBtZWFuKHJlY2FsbF9ib29zdGVkKSkpCgpwcmVjaXNpb25fcmVjYWxsX3RvdGFsX2Jvb3N0ZWQkTGFiZWxzIDwtIGMobGV2ZWxzKHRlc3RfZGF0YSRDbGFzcyksICJBdmVyYWdlIikKCnByaW50KGNvbmZ1c2lvbl9tYXRyaXhfYm9vc3RlZCkKcHJpbnQoYWNjdXJhY3lfYm9vc3RlZCkKcHJpbnQocGFzdGUwKCJUb3RhbCBFcnJvcjogIiwgcm91bmQodG90YWxfZXJyb3JfYm9vc3RlZCwgNCkpKQpwcmludChwcmVjaXNpb25fcmVjYWxsX3RvdGFsX2Jvb3N0ZWQpCmBgYAoKI0Rlc2ljaW9uIFRyZWUgdXNpbmcgcnBhcnQKYGBge3J9CmNyZWRpdF9kYXRhX21vZGVsIDwtIHRyYWluKENsYXNzIH4gLiwgZGF0YSA9IHRyYWluX2RhdGEsIG1ldGhvZCA9ICJycGFydCIsIHRyQ29udHJvbCA9IHRyYWluQ29udHJvbChtZXRob2QgPSAiY3YiLCBudW1iZXIgPSA1KSkKY3JlZGl0X2RhdGFfcHJlZCA8LSBwcmVkaWN0KGNyZWRpdF9kYXRhX21vZGVsLCB0ZXN0X2RhdGEpCmNyZWRpdF9kYXRhX2NtIDwtIGNvbmZ1c2lvbk1hdHJpeChjcmVkaXRfZGF0YV9wcmVkLCB0ZXN0X2RhdGEkQ2xhc3MpCmNyZWRpdF9kYXRhX2F1YyA8LSBjcmVkaXRfZGF0YV9jbSRvdmVyYWxsWzFdICoxMDAKY3JlZGl0X2RhdGFfYXVjCmNyZWRpdF9kYXRhX2FjYyA8LSBjcmVkaXRfZGF0YV9jbSRvdmVyYWxsWzRdICoxMDAKY3JlZGl0X2RhdGFfYWNjCmNhdCgiQWNjdXJhY3kgb2YgRGVjaXNpb24gVHJlZSBNb2RlbCIgLGNyZWRpdF9kYXRhX2FjYywiLiBTdW1tYXJ5IG9mIFJPQyBDdXJ2ZSAiLGNyZWRpdF9kYXRhX2F1YykKY3JlZGl0X2RhdGFfbW9kZWwKYGBgCiNLbm4KCmBgYHtyfQpzZXQuc2VlZCgxKQprbm5fY3JlZGl0X2NhcmQ8LSB0cmFpbihDbGFzcyB+IC4sIGRhdGEgPSB0cmFpbl9kYXRhLCB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kPSJjdiIsIG51bWJlcj0xMCwgY2xhc3NQcm9icyA9IFRSVUUsIHN1bW1hcnlGdW5jdGlvbiA9IHR3b0NsYXNzU3VtbWFyeSkgLG1ldGhvZCA9ICJrbm4iLG1ldHJpYyA9IlJPQyIpCmtubl9jcmVkaXRfY2FyZApgYGAKCmBgYHtyfQprbm5fcHJlZGljdGlvbnNfcHJvYj1wcmVkaWN0KGtubl9jcmVkaXRfY2FyZCwgdGVzdF9kYXRhLCB0eXBlPSJwcm9iIikKa25uX3ByZWRpY3Rpb25zPSBwcmVkaWN0aW9uKGtubl9wcmVkaWN0aW9uc19wcm9iJFllcyx0ZXN0X2RhdGEkQ2xhc3MpCnBlcmZvcm1hbmNlKGtubl9wcmVkaWN0aW9ucywgbWVhc3VyZSA9ICJhdWMiKQprbm5fcHJlZGljdGVkX2xhYmVscyA9IHByZWRpY3Qoa25uX2NyZWRpdF9jYXJkLCB0ZXN0X2RhdGEpCmNvbmZ1c2lvbk1hdHJpeChrbm5fcHJlZGljdGVkX2xhYmVscywgdGVzdF9kYXRhJENsYXNzICxwb3NpdGl2ZT0iWWVzIiwgbW9kZT0iZXZlcnl0aGluZyIpCmBgYAoKYGBge3J9CnBsb3Qoa25uX3ByZWRpY3Rpb25zX3Byb2IpCmBgYAoKQWNjdXJhY3kgOiAwLjk5MzgKCmBgYHtyfQpzZXQuc2VlZCgxKQprbm5fc21vdGU8LSB0cmFpbihDbGFzcyB+IC4sIGRhdGEgPSB0cmFpbl9kYXRhLHRyQ29udHJvbCAgPXRyYWluQ29udHJvbChtZXRob2Q9ImN2IiwgbnVtYmVyPTEwLCBjbGFzc1Byb2JzID0gVFJVRSwgc3VtbWFyeUZ1bmN0aW9uID0gdHdvQ2xhc3NTdW1tYXJ5LHNhbXBsaW5nPSJzbW90ZSIpICxtZXRob2QgPSAia25uIixtZXRyaWMgPSJST0MiKQprbm5fc21vdGUKYGBgCgpgYGB7cn0Ka25uX3Ntb3RlX3ByZWRpY3Rpb25zX3Byb2I9cHJlZGljdChrbm5fc21vdGUsIHRlc3RfZGF0YSwgdHlwZT0icHJvYiIpCmtubl9zbW90ZV9wcmVkaWN0aW9ucz0gcHJlZGljdGlvbihrbm5fc21vdGVfcHJlZGljdGlvbnNfcHJvYiRZZXMsdGVzdF9kYXRhJENsYXNzKQoKcGVyZm9ybWFuY2Uoa25uX3Ntb3RlX3ByZWRpY3Rpb25zLCBtZWFzdXJlID0gImF1YyIpCmtubl9zbW90ZV9wcmVkaWN0ZWRfbGFiZWxzID0gcHJlZGljdChrbm5fc21vdGUsIHRlc3RfZGF0YSkKY29uZnVzaW9uTWF0cml4KGtubl9zbW90ZV9wcmVkaWN0ZWRfbGFiZWxzLCB0ZXN0X2RhdGEkQ2xhc3MgLHBvc2l0aXZlPSJZZXMiLCBtb2RlPSJldmVyeXRoaW5nIikKYGBgCgpBY2N1cmFjeSA6IDAuOTkzOAoKCiNTVk0KCmBgYHtyfQpzZXQuc2VlZCgxKQoKc3ZtX2NyZWRpdDwtIHRyYWluKENsYXNzIH4gLiwgZGF0YSA9IHRyYWluX2RhdGEsCiAgICAgICAgICAgICB0ckNvbnRyb2wgPSB0cmFpbkNvbnRyb2wobWV0aG9kPSJjdiIsIG51bWJlcj0xMCwgY2xhc3NQcm9icyA9IFRSVUUsIHN1bW1hcnlGdW5jdGlvbiA9IHR3b0NsYXNzU3VtbWFyeSkgLG1ldGhvZCA9ICJzdm1MaW5lYXIiLG1ldHJpYyA9IlJPQyIpCnN2bV9jcmVkaXQKYGBgCmBgYHtyfQpzdm1fcHJlZGljdGlvbnNfcHJvYj1wcmVkaWN0KHN2bV9jcmVkaXQsIHRlc3RfZGF0YSwgdHlwZT0icHJvYiIpCnN2bV9wcmVkaWN0aW9ucz0gcHJlZGljdGlvbihzdm1fcHJlZGljdGlvbnNfcHJvYiRZZXMsdGVzdF9kYXRhJENsYXNzKQpzdm1fcHJlZGljdGlvbnNfcHJvYgoKcGVyZm9ybWFuY2Uoc3ZtX3ByZWRpY3Rpb25zLCBtZWFzdXJlID0gImF1YyIpCnN2bV9wcmVkaWN0ZWRfbGFiZWxzID0gcHJlZGljdChzdm1fY3JlZGl0LCB0ZXN0X2RhdGEpCmNvbmZ1c2lvbk1hdHJpeChzdm1fcHJlZGljdGVkX2xhYmVscywgdGVzdF9kYXRhJENsYXNzICxwb3NpdGl2ZT0iWWVzIiwgbW9kZT0iZXZlcnl0aGluZyIpCgpgYGAKIEFjY3VyYWN5IDogMC45NDIxCgojU1ZNIFNtb3RlCmBgYHtyfQpzZXQuc2VlZCgxKQpjdHJsPXRyYWluQ29udHJvbChtZXRob2Q9ImN2IiwgbnVtYmVyPTEwLCBjbGFzc1Byb2JzID0gVFJVRSwgc3VtbWFyeUZ1bmN0aW9uID0gdHdvQ2xhc3NTdW1tYXJ5LCAKc2FtcGxpbmc9InNtb3RlIikKc3ZtX3Ntb3RlPC10cmFpbihDbGFzcyB+IC4sIGRhdGEgPSB0cmFpbl9kYXRhLCBtZXRob2QgPSAic3ZtTGluZWFyIiwgdmVyYm9zZT1GQUxTRSwgbWV0cmljID0iUk9DIiwgCnRyQ29udHJvbD0gY3RybCkKc3ZtX3Ntb3RlCgpzdm1fc21vdGVfcHJlZGljdGlvbnNfcHJvYj1wcmVkaWN0KHN2bV9zbW90ZSwgdGVzdF9kYXRhLCB0eXBlPSJwcm9iIikKc3ZtX3Ntb3RlX3ByZWRpY3Rpb25zPSBwcmVkaWN0aW9uKHN2bV9zbW90ZV9wcmVkaWN0aW9uc19wcm9iJFllcyx0ZXN0X2RhdGEkQ2xhc3MpCgpwZXJmb3JtYW5jZShzdm1fc21vdGVfcHJlZGljdGlvbnMsIG1lYXN1cmUgPSAiYXVjIikKc3ZtX3Ntb3RlX3ByZWRpY3RlZF9sYWJlbHMgPSBwcmVkaWN0KHN2bV9zbW90ZSwgdGVzdF9kYXRhKQpjb25mdXNpb25NYXRyaXgoc3ZtX3Ntb3RlX3ByZWRpY3RlZF9sYWJlbHMsIHRlc3RfZGF0YSRDbGFzcyAscG9zaXRpdmU9IlllcyIsIG1vZGU9ImV2ZXJ5dGhpbmciKQpgYGAKCgpBY2N1cmFjeSA6IDAuOTQyMQoKCiMjIFNWTSBSYWRpYWwKCmBgYHtyfQpzZXQuc2VlZCgxKQpjcmVkaXRfcnN2bV9tb2RlbCA8LSB0cmFpbihDbGFzcyB+IC4sIGRhdGEgPSB0cmFpbl9kYXRhLCBtZXRob2QgPSAic3ZtUmFkaWFsIiwgdHJDb250cm9sID0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIsIG51bWJlciA9IDEwKSkKY3JlZGl0X3Jzdm1fbW9kZWwKY3JlZGl0X3Jzdm1fcHJlZCA8LSBwcmVkaWN0KGNyZWRpdF9yc3ZtX21vZGVsLCB0ZXN0X2RhdGEpCmNyZWRpdF9yc3ZtX2NtIDwtIGNvbmZ1c2lvbk1hdHJpeChjcmVkaXRfcnN2bV9wcmVkLCB0ZXN0X2RhdGEkQ2xhc3MpCmNyZWRpdF9yc3ZtX2F1YyA8LSBjcmVkaXRfcnN2bV9jbSRvdmVyYWxsWzFdICoxMDAKY3JlZGl0X3Jzdm1fYWNjIDwtIGNyZWRpdF9yc3ZtX2NtJG92ZXJhbGxbNF0gKjEwMApjcmVkaXRfcnN2bV9hdWMKCmBgYApBY2N1cmFjeSA5Ni44NSAKCmBgYHtyfQpjcmVkaXRfcnN2bV9tb2RlbApjcmVkaXRfcnN2bV9jbQpgYGAKCgojUmFuZG9tRm9yZXN0CgpgYGB7cn0Kc2V0LnNlZWQoMSkKY3JlZGl0X3JmX21vZGVsIDwtIHRyYWluKENsYXNzIH4gLiwgZGF0YSA9IHRyYWluX2RhdGEsIG1ldGhvZCA9ICJyZiIsaW1wb3J0YW5jZT0gVCxtZXRyaWMgPSAiUk9DIiwgdHJDb250cm9sID0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIsIG51bWJlciA9IDEwLCAgY2xhc3NQcm9icyA9IFRSVUUpKQpjcmVkaXRfcmZfbW9kZWwKYGBgCgpgYGB7cn0KY3JlZGl0X3JmX3ByZWQgPC0gcHJlZGljdChjcmVkaXRfcmZfbW9kZWwsIHRlc3RfZGF0YSkKY3JlZGl0X3JmX2NtIDwtIGNvbmZ1c2lvbk1hdHJpeChjcmVkaXRfcmZfcHJlZCwgdGVzdF9kYXRhJENsYXNzKQpjcmVkaXRfcmZfYXVjIDwtIGNyZWRpdF9yZl9jbSRvdmVyYWxsWzFdCmNyZWRpdF9yZl9hY2MgPC0gY3JlZGl0X3JmX2NtJG92ZXJhbGxbNF0KY3JlZGl0X3JmX2F1Ywp2YXJJbXAoY3JlZGl0X3JmX21vZGVsKQpgYGAKCmBgYHtyfQpjcmVkaXRfcmZfbW9kZWwKY3JlZGl0X3JmX2NtCmBgYAoKCiNHcmFkaWVudCBCb29zdGVkIFRyZWUgbW9kZWwKCmBgYHtyfQpzZXQuc2VlZCgxKQpjcmVkaXRfZ2JtX21vZGVsIDwtIHRyYWluKENsYXNzIH4gLiwgZGF0YSA9IHRyYWluX2RhdGEsIG1ldGhvZCA9ICJnYm0iLCB0ckNvbnRyb2wgID0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIsIG51bWJlciA9IDEwKSkKY3JlZGl0X2dibV9wcmVkaWN0aW9ucyA8LSBwcmVkaWN0KGNyZWRpdF9nYm1fbW9kZWwsdGVzdF9kYXRhKQpjcmVkaXRfZ2JtX2NtIDwtIGNvbmZ1c2lvbk1hdHJpeChjcmVkaXRfZ2JtX3ByZWRpY3Rpb25zLCB0ZXN0X2RhdGEkQ2xhc3MpCmNyZWRpdF9nYm1fYXVjIDwtIGNyZWRpdF9nYm1fY20kb3ZlcmFsbFsxXQpjcmVkaXRfZ2JtX2F1YwpjcmVkaXRfZ2JtX2FjYyA8LSBjcmVkaXRfZ2JtX2NtJG92ZXJhbGxbNF0KY3JlZGl0X2dibV9hY2MKYGBgCgpBY2N1cmFjeSA6IDAuOTY5OCAKYGBge3J9CmNyZWRpdF9nYm1fbW9kZWwKY3JlZGl0X2dibV9jbQpgYGAKCiMjIFhHQm9vc3QKCmBgYHtyfQpzZXQuc2VlZCgxKQpjcmVkaXRfeGdiX21vZGVsIDwtIHRyYWluKENsYXNzIH4gLiwgZGF0YSA9IHRyYWluX2RhdGEsIG1ldGhvZCA9ICJ4Z2JUcmVlIiwgdHJDb250cm9sID0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJjdiIsIG51bWJlciA9IDEwKSx2ZXJib3NpdHkgPSAwKQpjcmVkaXRfeGdiX21vZGVsCmNyZWRpdF94Z2JfcHJlZCA8LSBwcmVkaWN0KGNyZWRpdF94Z2JfbW9kZWwsIHRlc3RfZGF0YSkKY3JlZGl0X3hnYl9jbSA8LSBjb25mdXNpb25NYXRyaXgoY3JlZGl0X3hnYl9wcmVkLCB0ZXN0X2RhdGEkQ2xhc3MpCmNyZWRpdF94Z2JfYXVjIDwtIGNyZWRpdF94Z2JfY20kb3ZlcmFsbFsxXSAqMTAwCmNyZWRpdF94Z2JfYWNjIDwtIGNyZWRpdF94Z2JfY20kb3ZlcmFsbFs0XSAqMTAwCmNyZWRpdF94Z2JfYXVjCmBgYAoKYGBge3J9CmNyZWRpdF94Z2JfbW9kZWwKY3JlZGl0X3hnYl9jbQpgYGAKCgpgYGB7cn0KdHJhaW5fZGF0YSRDbGFzcz1hcy5udW1lcmljKHRyYWluX2RhdGEkQ2xhc3MpLTEKdGVzdF9kYXRhJENsYXNzPWFzLm51bWVyaWModGVzdF9kYXRhJENsYXNzKS0xCmBgYAoKCmBgYHtyfQpoZWFkKHRlc3RfZGF0YSkKaGVhZCh0cmFpbl9kYXRhKQpgYGAKYGBge3J9Cm5vcm1hbGl6ZSA8LSBmdW5jdGlvbih4KSB7CiAgcmV0dXJuICgoeCAtIG1pbih4KSkgLyAobWF4KHgpIC0gbWluKHgpKSkKfQpzY2FsZS5jb2xzIDwtIGMoIlYzIiwiVjQiLCJWNyIsIlYxMCIsIlYxMiIsIlYxNCIsIlYxNiIsIlYxNyIpCnRyYWluX2RhdGFbc2NhbGUuY29sc10gPC0gbGFwcGx5KHRyYWluX2RhdGFbc2NhbGUuY29sc10sIG5vcm1hbGl6ZSkKdGVzdF9kYXRhW3NjYWxlLmNvbHNdIDwtIGxhcHBseSh0ZXN0X2RhdGFbc2NhbGUuY29sc10sIG5vcm1hbGl6ZSkKdHJhaW5fZGF0YQp0ZXN0X2RhdGEKYGBgCiNOZXVyYWwgTmV0d29yayBNb2RlbAoKYGBge3J9CnNldC5zZWVkKDEpCmNyZWRpdF90cmFpbl9pZGV4MTwtIGNyZWF0ZURhdGFQYXJ0aXRpb24odHJhaW5fZGF0YSRDbGFzcywgcCA9IDAuOSwgbGlzdCA9IEZBTFNFKQoKY3JlZGl0X2NhcmRfdHJhaW4gPSB0cmFpbl9kYXRhW2NyZWRpdF90cmFpbl9pZGV4MSxdCmNyZWRpdF9jYXJkX3ZhbGlkID0gdHJhaW5fZGF0YVstY3JlZGl0X3RyYWluX2lkZXgxLF0KCmNyZWRpdF9jYXJkX3RyYWluX3ggPC0gY3JlZGl0X2NhcmRfdHJhaW5bLCAtOV0KY3JlZGl0X2NhcmRfdHJhaW5feSA8LSBjcmVkaXRfY2FyZF90cmFpblssIDldCmNyZWRpdF9jYXJkX3ZhbGlkX3ggPC0gY3JlZGl0X2NhcmRfdmFsaWRbLC05XQpjcmVkaXRfY2FyZF92YWxpZF95IDwtIGNyZWRpdF9jYXJkX3ZhbGlkWywgOV0KY3JlZGl0X2NhcmRfdGVzdF94IDwtdGVzdF9kYXRhWywgLTldCmNyZWRpdF9jYXJkX3Rlc3RfeSA8LSB0ZXN0X2RhdGFbLCA5XQpgYGAKCgoKYGBge3J9CmxpYnJhcnkoa2VyYXMpCnNldC5zZWVkKDEpCgptb2RlbCA8LSBrZXJhc19tb2RlbF9zZXF1ZW50aWFsKCkgJT4lCiAgbGF5ZXJfZGVuc2UodW5pdHMgPSA2NCwgYWN0aXZhdGlvbiA9ICJyZWx1IiwgaW5wdXRfc2hhcGUgPSBhcy5tYXRyaXgoZGltKGNyZWRpdF9jYXJkX3RyYWluX3gpWzJdKSkgJT4lCiAgbGF5ZXJfZGVuc2UodW5pdHMgPSA2NCwgYWN0aXZhdGlvbiA9ICJyZWx1IikgJT4lCiAgbGF5ZXJfZGVuc2UodW5pdHMgPSAxLCBhY3RpdmF0aW9uID0gInNpZ21vaWQiKQoKCm1vZGVsICU+JSBjb21waWxlKAogIGxvc3MgPSAiYmluYXJ5X2Nyb3NzZW50cm9weSIsCiAgb3B0aW1pemVyID0gb3B0aW1pemVyX2FkYW0obGVhcm5pbmdfcmF0ZSA9IDAuMDAxKSwKICBtZXRyaWNzID0gImFjY3VyYWN5IgoKICAKKQoKaGlzdG9yeSA8LSBtb2RlbCAlPiUgZml0KAogIGFzLm1hdHJpeChjcmVkaXRfY2FyZF90cmFpbl94KSwKICBjcmVkaXRfY2FyZF90cmFpbl95LAogIGJhdGNoX3NpemUgPSA1MCwKICBlcG9jaHMgPSAyMCwKICB2ZXJib3NlID0gMiwKICB2YWxpZGF0aW9uX2RhdGEgPSBsaXN0KGFzLm1hdHJpeChjcmVkaXRfY2FyZF92YWxpZF94KSwgY3JlZGl0X2NhcmRfdmFsaWRfeSkKKQoKcHJlZGljdGVkX3Byb2JzIDwtIG1vZGVsICU+JSBwcmVkaWN0KGFzLm1hdHJpeChjcmVkaXRfY2FyZF90ZXN0X3gpKQpzdHIocHJlZGljdGVkX3Byb2JzKQoKYGBgCmBgYHtyfQoKYW5uX3ByZWRpY3Rpb25zPSBwcmVkaWN0aW9uKHByZWRpY3RlZF9wcm9icyxjcmVkaXRfY2FyZF90ZXN0X3kpCmFubl9wcmVkaWN0aW9ucwpwZXJmb3JtYW5jZShhbm5fcHJlZGljdGlvbnMsIG1lYXN1cmUgPSAiYXVjIilAeS52YWx1ZXMKYGBgCgpgYGB7cn0KcGxvdChoaXN0b3J5KQpgYGAKCmBgYHtyfQpwcmVkaWN0ZWQubGFiZWxzID0gZmFjdG9yKGlmZWxzZShwcmVkaWN0ZWRfcHJvYnM+MC41LCAiMSIsICIwIikpCmNvbmZ1c2lvbk1hdHJpeChwcmVkaWN0ZWQubGFiZWxzLCBhcy5mYWN0b3IoY3JlZGl0X2NhcmRfdGVzdF95KSxtb2RlPSJldmVyeXRoaW5nIiwgcG9zaXRpdmU9IjEiKQoKYGBgClRoZSBBY2N1cmFjeSBvZiBOZXVyYWwgTmV0d29yayBNb2RlbCBpcyA5MC43CgpgYGB7cn0KbW9kZWwKYGBgCgojTmV1cmFsIE5ldHdvcmsgTW9kZWwgV2l0aCBDbGFzc3dlaWdodHMuCmBgYHtyfQpsaWJyYXJ5KGtlcmFzKQoKbW9kZWxfd2VpZ2h0IDwtIGtlcmFzX21vZGVsX3NlcXVlbnRpYWwoKSAlPiUKICBsYXllcl9kZW5zZSh1bml0cyA9IDY0LCBhY3RpdmF0aW9uID0gInJlbHUiLCBpbnB1dF9zaGFwZSA9IGFzLm1hdHJpeChkaW0oY3JlZGl0X2NhcmRfdHJhaW5feClbMl0pKSAlPiUKICBsYXllcl9kZW5zZSh1bml0cyA9IDY0LCBhY3RpdmF0aW9uID0gInJlbHUiKSAlPiUKICBsYXllcl9kZW5zZSh1bml0cyA9IDEsIGFjdGl2YXRpb24gPSAic2lnbW9pZCIpCgoKbW9kZWxfd2VpZ2h0ICU+JSBjb21waWxlKAogIGxvc3MgPSAiYmluYXJ5X2Nyb3NzZW50cm9weSIsCiAgb3B0aW1pemVyID0gb3B0aW1pemVyX2FkYW0obGVhcm5pbmdfcmF0ZSA9IDAuMDAxKSwKICBtZXRyaWNzID0gImFjY3VyYWN5IgopCgpoaXN0b3J5X21vZGVsIDwtIG1vZGVsX3dlaWdodCAlPiUgZml0KAogIGFzLm1hdHJpeChjcmVkaXRfY2FyZF90cmFpbl94KSwKICBjcmVkaXRfY2FyZF90cmFpbl95LAogIGJhdGNoX3NpemUgPSA1MCwKICBlcG9jaHMgPSAyMCwKICB2ZXJib3NlID0gMiwKICB2YWxpZGF0aW9uX2RhdGEgPSBsaXN0KGFzLm1hdHJpeChjcmVkaXRfY2FyZF92YWxpZF94KSwgY3JlZGl0X2NhcmRfdmFsaWRfeSksY2xhc3Nfd2VpZ2h0PWxpc3QoIjAiLCAiMSIpCgopCmBgYAoKYGBge1J9CnBsb3QoaGlzdG9yeV9tb2RlbCkKYGBgCgpgYGB7Un0KcHJlZGljdGVkX3Byb2JzX3dlaWdodCA8LSBtb2RlbF93ZWlnaHQgJT4lIHByZWRpY3QoYXMubWF0cml4KGNyZWRpdF9jYXJkX3Rlc3RfeCkpCnN0cihwcmVkaWN0ZWRfcHJvYnNfd2VpZ2h0KQoKYGBgCgoKYGBge1J9CmFubl9wcmVkaWN0aW9uc193ZWlnaHQ9IHByZWRpY3Rpb24ocHJlZGljdGVkX3Byb2JzLGNyZWRpdF9jYXJkX3Rlc3RfeSkKYW5uX3ByZWRpY3Rpb25zX3dlaWdodApwZXJmb3JtYW5jZShhbm5fcHJlZGljdGlvbnNfd2VpZ2h0LCBtZWFzdXJlID0gImF1YyIpQHkudmFsdWVzCgpgYGAKCmBgYHtSfQpyYW5nZShjcmVkaXRfY2FyZF90ZXN0X3gpCmBgYAoKYGBge1J9CnByZWRpY3RlZC5sYWJlbHNfd2VpZ2h0ID0gZmFjdG9yKGlmZWxzZShwcmVkaWN0ZWRfcHJvYnNfd2VpZ2h0PjAuNSwgIjEiLCAiMCIpKQpjb25mdXNpb25NYXRyaXgocHJlZGljdGVkLmxhYmVsc193ZWlnaHQsIGFzLmZhY3RvcihjcmVkaXRfY2FyZF90ZXN0X3kpLG1vZGU9ImV2ZXJ5dGhpbmciLCBwb3NpdGl2ZT0iMSIpCmBgYAoKCiNTdW1tYXJ5IG9mIEFjY3VyYWN5CgpSYW5kb20gRm9yZXN0OiA5OS45OSUKCkRlY2lzaW9uVHJlZU1vZGVsIHVzaW5nIEM1MDogOTkuOTclCgpYR0Jvb3N0OiA5OS45Mzc5NyUKCktOTiBTbW90ZTogOTkuMzglCgpLTk46IDk5LjM4JQoKR3JhZGllbnQgQm9vc3RlZCBUcmVlIG1vZGVsOiA5Ny4xNyUKClNWTSBSYWRpY2FsOiA5Ny4wNCUKCk5ldXJhbE5ldHdvcmsgd2l0aCBjbGFzcyB3ZWlnaHQ6IDkyLjY0JQoKRGVjaXNpb25UcmVlTW9kZWwgdXNpbmcgcnBhcnQ6IDkzLjUwMTk1JQoKU1ZNIFNtb3RlOiA5NC4yMSUKClNWTTogOTQuMjElCgpOZXVyYWwgTmV0d29yayBNb2RlbDogOTAuNyUKCgo=